From 4171bb3f4f1c8f2ad69695f7174e7bba62725003 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 00:34:10 -0500 Subject: [PATCH 01/26] docs: record M0 design decisions --- AGENTS.md | 6 +++--- ROADMAP.md | 20 +++++++++++--------- 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0a430f0..f2eed26 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -18,7 +18,7 @@ There is no code yet. Add the build, test and publish commands here when M0 land The product is only as good as these rules. Never break them, not even in debug modes or dev tooling. -- **Read-only, always.** A check is one `SELECT` against catalogs and statistics views. No DDL or DML, and no functions with side effects: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_*`, `pg_drop_*`, `nextval`, `setval`. Checks run under the session guards (`default_transaction_read_only`, `statement_timeout`, `lock_timeout`) inside a `READ ONLY` transaction. Never weaken or bypass them. +- **Read-only, always.** A check is one `SELECT` against catalogs and statistics views. No DDL or DML, and no functions with side effects: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_*`, `pg_drop_*`, `pg_advisory_*`, `nextval`, `setval`, `set_config`, `txid_current`. Every statement pgcheckup sends runs inside `BEGIN READ ONLY` with `SET LOCAL statement_timeout` and `lock_timeout`, then rolls back. Never set anything for the whole session, because behind a transaction pooler it reaches the app's connections. Never weaken or bypass these guards. - **Fixes are text.** pgcheckup prints fix SQL and never executes it. - **Least privilege.** No check needs more than `pg_monitor`. Never require superuser or `rds_superuser`. If the role lacks a privilege, the check is skipped with the reason. It is never an error. - **No network beyond the Postgres connection.** No telemetry, update checks, crash reporting or remote lookups. Data such as end-of-life dates ships inside the release. @@ -29,10 +29,10 @@ The product is only as good as these rules. Never break them, not even in debug ## Checks - One folder per check: `checks//check.sql`, `check.md`, `fixtures/fires.sql` and `fixtures/healthy.sql`. The shape is in the `ROADMAP.md` decisions. -- `check.sql` is one read-only query that returns the fixed shape. Thresholds come in as parameters and are never hard-coded. +- `check.sql` is one read-only query that returns values, never prose. The wording lives in the `message` and `fix` templates in `check.md`. Thresholds come in as `@name` parameters and are never hard-coded. - Compute ages and durations in SQL from the server's `now()`, not the client's clock. - `check.md` has **What breaks**, **Fix** and **Seen in** sections. Every check has at least one **Seen in** link to a public incident or the Postgres docs. Never cite anything a reader can't open. -- Both fixtures are required. `fires.sql` is the positive control, so a check without one isn't done. A fixture may lower a threshold when the real condition can't be reproduced at scale. +- Both fixtures are required. `fires.sql` is the positive control, so a check without one isn't done. A fixture may lower a threshold (`-- threshold name = value`) when the real condition can't be reproduced at scale. - Check ids are kebab-case and stable, because baselines and ignore lists depend on them. Renaming one is a breaking change that needs a decision in `ROADMAP.md`. - Severity: `critical` can take the database down or lose data soon. `warning` is heading there, or removes a safety net. `info` is housekeeping. Don't inflate severity. - A check declares its minimum Postgres version and the providers where it is skipped. Every check is tested on every supported version. diff --git a/ROADMAP.md b/ROADMAP.md index a7a25fc..5fb1f8d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -6,25 +6,27 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab - **Name.** The idea started as "Postgres Doctor". `pgdoctor` is already an existing Go project, so this one is pgcheckup. PostgresAI's `postgres-checkup` is unrelated, and the README says so. - **Position.** Production readiness for teams without a DBA. It has fewer checks than pg-healthcheck or pgdoctor, and each one maps to a failure that causes outages and is explained in plain words with its fix. It is not a DBA toolkit. -- **Read-only by construction.** A scan opens one connection with `default_transaction_read_only = on`, `application_name = pgcheckup`, `statement_timeout` (5 s) and `lock_timeout` (1 s). Every check runs inside its own `READ ONLY` transaction. The lock timeout stops a scan from queueing behind a migration's lock and then blocking the traffic behind it. pgcheckup prints fixes and never runs them. -- **Least privilege.** No check needs more than `pg_monitor`. Each check declares what it needs, and if the role doesn't have it, the check is skipped with the reason. `pgcheckup grant` prints the SQL for a checkup role. +- **Read-only by construction.** Every statement pgcheckup sends runs in its own transaction: `BEGIN READ ONLY`, `SET LOCAL statement_timeout = '5s'`, `SET LOCAL lock_timeout = '1s'`, the query, then `ROLLBACK`. Only `application_name = pgcheckup` is set for the whole session. Behind a transaction-mode pooler such as PgBouncer, a session-level `SET` would reach the app's next transaction, and the `options` connection parameter is rejected or dropped. The lock timeout stops a scan from queueing behind a migration's lock and then blocking the traffic behind it. pgcheckup prints fixes and never runs them. +- **Least privilege.** No check needs more than `pg_monitor`. Each check declares what it needs, and if the role doesn't have it, the check is skipped with the reason. `pgcheckup grant` prints the SQL for a checkup role, including `ALTER ROLE … SET default_transaction_read_only = on`, so the role is read-only outside pgcheckup too. - **SQL only.** pgcheckup talks only to Postgres. Provider settings that SQL can see (such as `rds.force_ssl`) are in scope. Checks that need a cloud API (RDS backups, deletion protection, encryption at rest) are not. - **Managed providers.** pgcheckup detects RDS/Aurora, Cloud SQL, Azure Database for PostgreSQL, Supabase and Neon from SQL. A check can list providers where it doesn't apply or can't run, and it shows as skipped there, with the reason. - **Checks are SQL plus Markdown**, one folder per check under `checks//`: - - `check.sql`: one read-only query that returns findings in a fixed shape (`subject`, `severity`, `detail`, plus named values for the message); - - `check.md`: frontmatter (id, title, category, default severity, thresholds, minimum Postgres version, required privileges, providers to skip) and a body with **What breaks**, **Fix** and **Seen in** (links to public incidents or the Postgres docs); + - `check.sql`: one read-only statement, starting with `SELECT` or `WITH`, that returns a row per finding: `subject`, an optional `severity` that overrides the default, and the named values the templates use; + - `check.md`: frontmatter (id, title, category, default severity, minimum Postgres version, required privileges, providers to skip, thresholds, and the `message` and `fix` templates) and a body with **What breaks**, **Fix** and **Seen in** (links to public incidents or the Postgres docs); - `fixtures/fires.sql` and `fixtures/healthy.sql`: setup scripts. The check must report on the first and stay quiet on the second. - Checks are embedded in the binary at build time, and the build fails on invalid frontmatter. -- **Thresholds** live in each check's frontmatter, with defaults. `check.sql` reads them as parameters, so a fixture can lower a threshold when the real condition can't be reproduced at full scale (wraparound, for example). Overrides from a config file come in v0.2. + A source generator compiles the checks into the binary, so nothing is parsed at runtime. The build fails on invalid frontmatter or templates, a missing section or fixture, an unused or unknown threshold, and SQL that calls a function with side effects that a `READ ONLY` transaction still allows (`pg_terminate_backend`, `pg_cancel_backend`, advisory locks, `txid_current` and others). Npgsql's SQL rewriting is off, so the server rejects a second statement. +- **Messages are templates**, so numbers read the same in every check. `{name}` inserts a value, and intervals print as "3 days". `{name:count}` and `{name:bytes}` print as "1.61 billion" and "48 GB". A `[…]` section is left out when a value in it is NULL, so one template covers older Postgres versions. `check.sql` quotes names for fix SQL with `quote_ident` and `quote_literal`. JSON output gets the raw values. +- **Thresholds** live in each check's frontmatter, with defaults in Postgres units (`1GB`, `30min`, `1h`). `check.sql` reads them as `@name` parameters, which become `$1`, `$2`… at build time. A fixture can lower one with a `-- threshold name = value` line when the real condition can't be reproduced at full scale (wraparound, for example). Overrides from a config file come in v0.2. - **Ages and durations** are computed in SQL from the server's clock, so a skewed client clock can't change a finding. - **Postgres versions:** every community-supported major (14 to 18 today), each tested in CI with Testcontainers. When a major reaches end of life, it moves to best effort, and scanning it reports the end of life as a finding. +- **Fixture tests** give each fixture a fresh Testcontainers Postgres, because slots, prepared transactions and roles belong to the whole server. The fixture's connection stays open until the check has run, so a fixture can hold a transaction open. The check runs through the same code as `scan`, as a role with only `pg_monitor`. - **Output:** terminal (default), `--format json` and `--format markdown` in v0.1. The JSON has a `schema` version. The HTML report comes in v0.2. - **Exit codes:** 0 when no finding reaches `--fail-on` (default `critical`), 1 when one does, and 2 when the scan couldn't run. Skipped checks never fail a scan. - **Severity:** `critical` can take the database down or lose data soon. `warning` is heading there, or removes a safety net. `info` is housekeeping. - **Connection:** a `postgres://` URL, a key-value connection string, or the standard `PG*` environment variables and `.pgpass`. Docs keep passwords off the command line. - **What output may contain.** Findings name database objects (tables, slots, roles), settings, process IDs and durations. They never include query text, row data, passwords or client addresses. - **No network** beyond the Postgres connection. No telemetry and no update check. End-of-life dates ship inside each release. -- **Runtime:** .NET 10 (LTS) with NativeAOT, and Npgsql through `NpgsqlSlimDataSourceBuilder`. Trim and AOT warnings are errors. +- **Runtime:** .NET 10 (LTS) with NativeAOT, and Npgsql through `NpgsqlSlimDataSourceBuilder`. Trim and AOT warnings are errors. The CLI uses System.CommandLine, and tests use xUnit v3 and Testcontainers. - **Distribution:** NativeAOT binaries for linux-x64, linux-arm64, osx-arm64 and win-x64 on GitHub Releases, with SHA-256 checksums, plus a container image on GHCR for CI. NativeAOT can't cross-compile between operating systems, so releases build on a runner matrix. A `dotnet tool` package comes in v0.2. - **The paid path comes later.** The CLI stays free and complete. A one-time audit report, hosted monitoring and a team dashboard are listed under Later and get built only if public signals show demand. They would be separate code. - **Brand** is option 1A, "Scan": stacked layers read by a single probe line. The wordmark is Bricolage Grotesque SemiBold (optical size 34), converted to vector paths, with "pg" in Postgres blue (`#336791`, or `#5B9BD5` on dark). The assets are in `docs/brand/`, with `-dark` files for dark backgrounds. @@ -46,7 +48,7 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab ## M1: Engine and check catalog -- [ ] Engine: session guards, server version and provider detection, and a privilege probe, followed by every applicable check in its own read-only transaction. A check that errors or times out is reported as errored, and the others still run. +- [ ] Engine: server version and provider detection, and a privilege probe, followed by every applicable check. A check that errors or times out is reported as errored, and the others still run. - [ ] Provider detection, tested by simulating each provider's roles and settings in fixtures. - [ ] Desk research for the catalog: go through public Postgres postmortems (danluu/post-mortems, engineering blogs) and DBA Stack Exchange, list the failures that recur, and adjust the table below to match. Every check gets at least one **Seen in** link. - [ ] The v0.1 checks: @@ -69,7 +71,7 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab | `postgres-eol` | Major versions past their end-of-life date | - [ ] `pgcheckup explain ` prints the check's note. `pgcheckup list` shows every check with its category and minimum version. -- [ ] `pgcheckup grant` prints SQL for a least-privilege checkup role (`pg_monitor` plus `CONNECT`). +- [ ] `pgcheckup grant` prints SQL for a least-privilege checkup role: `pg_monitor`, `CONNECT`, and `default_transaction_read_only = on` for the role. - [ ] `--format json` and `--format markdown`. The JSON shape is documented, with `"schema": 1`. **Done when:** every check's fixtures pass on Postgres 14 to 18, and a scan as a role with only `pg_monitor` either runs or skips (with a reason) every check, with no errors. From 07d19279c83090e5739bf5debaac4b15525dcee0 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 00:36:39 -0500 Subject: [PATCH 02/26] build: scaffold .NET 10 solution with NativeAOT --- Directory.Build.props | 12 ++++++++++++ Directory.Packages.props | 16 ++++++++++++++++ global.json | 6 ++++++ pgcheckup.slnx | 9 +++++++++ .../Pgcheckup.Checks.Generator.csproj | 12 ++++++++++++ src/Pgcheckup/Pgcheckup.csproj | 13 +++++++++++++ src/Pgcheckup/Program.cs | 4 ++++ tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj | 19 +++++++++++++++++++ 8 files changed, 91 insertions(+) create mode 100644 Directory.Build.props create mode 100644 Directory.Packages.props create mode 100644 global.json create mode 100644 pgcheckup.slnx create mode 100644 src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj create mode 100644 src/Pgcheckup/Pgcheckup.csproj create mode 100644 src/Pgcheckup/Program.cs create mode 100644 tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj diff --git a/Directory.Build.props b/Directory.Build.props new file mode 100644 index 0000000..8f38ccc --- /dev/null +++ b/Directory.Build.props @@ -0,0 +1,12 @@ + + + net10.0 + latest + enable + enable + true + true + 0.0.0 + false + + diff --git a/Directory.Packages.props b/Directory.Packages.props new file mode 100644 index 0000000..2b4b12b --- /dev/null +++ b/Directory.Packages.props @@ -0,0 +1,16 @@ + + + true + + + + + + + + + + + + + diff --git a/global.json b/global.json new file mode 100644 index 0000000..512142d --- /dev/null +++ b/global.json @@ -0,0 +1,6 @@ +{ + "sdk": { + "version": "10.0.100", + "rollForward": "latestFeature" + } +} diff --git a/pgcheckup.slnx b/pgcheckup.slnx new file mode 100644 index 0000000..e3e7bd8 --- /dev/null +++ b/pgcheckup.slnx @@ -0,0 +1,9 @@ + + + + + + + + + diff --git a/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj b/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj new file mode 100644 index 0000000..c5d6335 --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj @@ -0,0 +1,12 @@ + + + netstandard2.0 + disable + true + true + + + + + + diff --git a/src/Pgcheckup/Pgcheckup.csproj b/src/Pgcheckup/Pgcheckup.csproj new file mode 100644 index 0000000..1f8c0ab --- /dev/null +++ b/src/Pgcheckup/Pgcheckup.csproj @@ -0,0 +1,13 @@ + + + Exe + pgcheckup + Pgcheckup + true + false + + + + + + diff --git a/src/Pgcheckup/Program.cs b/src/Pgcheckup/Program.cs new file mode 100644 index 0000000..e383c9e --- /dev/null +++ b/src/Pgcheckup/Program.cs @@ -0,0 +1,4 @@ +using System.CommandLine; + +var root = new RootCommand("Checks a PostgreSQL database for the problems that cause outages."); +return root.Parse(args).Invoke(); diff --git a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj new file mode 100644 index 0000000..847ac83 --- /dev/null +++ b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj @@ -0,0 +1,19 @@ + + + Exe + false + + + + + + + + + + + + + + + From 116a6a48c6b5d4f5d7c79f9d5b44d5fd242983d1 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 00:48:56 -0500 Subject: [PATCH 03/26] build: run tests on Microsoft.Testing.Platform --- Directory.Packages.props | 2 -- global.json | 3 +++ tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj | 2 -- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/Directory.Packages.props b/Directory.Packages.props index 2b4b12b..a03466f 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -6,11 +6,9 @@ - - diff --git a/global.json b/global.json index 512142d..1d364c6 100644 --- a/global.json +++ b/global.json @@ -2,5 +2,8 @@ "sdk": { "version": "10.0.100", "rollForward": "latestFeature" + }, + "test": { + "runner": "Microsoft.Testing.Platform" } } diff --git a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj index 847ac83..c4674b1 100644 --- a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj +++ b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj @@ -4,9 +4,7 @@ false - - From 3dff171eee05c079c95c2cf91d410cade2086e39 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 00:48:57 -0500 Subject: [PATCH 04/26] feat(checks): render finding templates with consistent units --- src/Pgcheckup/Checks/CheckDefinition.cs | 34 +++++ src/Pgcheckup/Checks/Template.cs | 83 ++++++++++++ src/Pgcheckup/Checks/ValueText.cs | 94 +++++++++++++ .../Checks/TemplateRenderTests.cs | 125 ++++++++++++++++++ 4 files changed, 336 insertions(+) create mode 100644 src/Pgcheckup/Checks/CheckDefinition.cs create mode 100644 src/Pgcheckup/Checks/Template.cs create mode 100644 src/Pgcheckup/Checks/ValueText.cs create mode 100644 tests/Pgcheckup.Tests/Checks/TemplateRenderTests.cs diff --git a/src/Pgcheckup/Checks/CheckDefinition.cs b/src/Pgcheckup/Checks/CheckDefinition.cs new file mode 100644 index 0000000..5739efe --- /dev/null +++ b/src/Pgcheckup/Checks/CheckDefinition.cs @@ -0,0 +1,34 @@ +namespace Pgcheckup.Checks; + +// Ordered so that a higher value is more severe. +public enum Severity +{ + Info, + Warning, + Critical, +} + +public enum ThresholdKind +{ + Bytes, + Duration, + Integer, + Number, +} + +// Durations are held in microseconds, as the generator parsed them. +public sealed record Threshold(string Name, ThresholdKind Kind, decimal Value, string Text); + +public sealed record CheckDefinition( + string Id, + string Title, + string Category, + Severity Severity, + int MinVersion, + IReadOnlyList Privileges, + IReadOnlyList SkipOn, + IReadOnlyList Thresholds, + string Sql, + Template Message, + Template Fix, + string Note); diff --git a/src/Pgcheckup/Checks/Template.cs b/src/Pgcheckup/Checks/Template.cs new file mode 100644 index 0000000..5579612 --- /dev/null +++ b/src/Pgcheckup/Checks/Template.cs @@ -0,0 +1,83 @@ +using System.Text; + +namespace Pgcheckup.Checks; + +public enum ValueFormat +{ + Default, + Bytes, + Count, +} + +public abstract record TemplatePart; + +public sealed record TextPart(string Text) : TemplatePart; + +public sealed record ValuePart(string Name, ValueFormat Format) : TemplatePart; + +public sealed record SectionPart(IReadOnlyList Parts) : TemplatePart; + +public sealed class TemplateException(string message) : Exception(message); + +public sealed class Template(IReadOnlyList parts) +{ + public IReadOnlyList Parts { get; } = parts; + + public string Render(IReadOnlyDictionary values) + { + var text = new StringBuilder(); + foreach (var part in Parts) + { + switch (part) + { + case TextPart t: + text.Append(t.Text); + break; + + case ValuePart v: + text.Append(Format(v, values) ?? throw new TemplateException($"The query returned NULL for {v.Name}, which the template needs.")); + break; + + case SectionPart s: + var section = new StringBuilder(); + var complete = true; + foreach (var inner in s.Parts) + { + var rendered = inner switch + { + TextPart t => t.Text, + ValuePart v => Format(v, values), + _ => throw new TemplateException("A template section can't contain another section."), + }; + + if (rendered == null) + { + complete = false; + break; + } + + section.Append(rendered); + } + + if (complete) + { + text.Append(section); + } + + break; + } + } + + return text.ToString(); + } + + private static string? Format(ValuePart part, IReadOnlyDictionary values) + { + if (!values.TryGetValue(part.Name, out var value)) + { + throw new TemplateException($"The template uses {part.Name}, but the query returned no column with that name."); + } + + return value == null ? null : ValueText.Format(value, part.Format); + } +} diff --git a/src/Pgcheckup/Checks/ValueText.cs b/src/Pgcheckup/Checks/ValueText.cs new file mode 100644 index 0000000..a914a12 --- /dev/null +++ b/src/Pgcheckup/Checks/ValueText.cs @@ -0,0 +1,94 @@ +using System.Globalization; + +namespace Pgcheckup.Checks; + +public static class ValueText +{ + private static readonly string[] ByteUnits = ["bytes", "kB", "MB", "GB", "TB", "PB"]; + private static readonly string[] CountUnits = ["million", "billion", "trillion"]; + + public static string Format(object value, ValueFormat format) => format switch + { + ValueFormat.Bytes => Bytes(ToDecimal(value)), + ValueFormat.Count => Count(ToDecimal(value)), + _ => value switch + { + TimeSpan span => Duration(span), + DateTime time => time.ToUniversalTime().ToString("yyyy-MM-dd HH:mm", CultureInfo.InvariantCulture) + " UTC", + bool flag => flag ? "on" : "off", + IFormattable number => number.ToString(null, CultureInfo.InvariantCulture), + _ => value.ToString() ?? "", + }, + }; + + // Postgres's size units (1024-based, as in pg_size_pretty), with three significant digits. + public static string Bytes(decimal bytes) + { + if (bytes < 1024) + { + return bytes == 1 ? "1 byte" : $"{Significant(bytes)} bytes"; + } + + var unit = 0; + var value = bytes; + while (unit < ByteUnits.Length - 1 && (value >= 1024 || Round(value) >= 1024)) + { + value /= 1024; + unit++; + } + + return $"{Significant(value)} {ByteUnits[unit]}"; + } + + public static string Count(decimal count) + { + if (count < 1_000_000) + { + return Math.Round(count).ToString("#,0", CultureInfo.InvariantCulture); + } + + var unit = 0; + var value = count / 1_000_000; + while (unit < CountUnits.Length - 1 && (value >= 1000 || Round(value) >= 1000)) + { + value /= 1000; + unit++; + } + + return $"{Significant(value)} {CountUnits[unit]}"; + } + + public static string Duration(TimeSpan span) + { + if (span < TimeSpan.Zero) + { + span = TimeSpan.Zero; + } + + return span switch + { + { TotalDays: >= 1 } => Plural((long)span.TotalDays, "day"), + { TotalHours: >= 1 } => Plural((long)span.TotalHours, "hour"), + { TotalMinutes: >= 1 } => Plural((long)span.TotalMinutes, "minute"), + _ => Plural((long)span.TotalSeconds, "second"), + }; + } + + private static string Plural(long n, string unit) => n == 1 ? $"1 {unit}" : $"{n} {unit}s"; + + private static decimal Round(decimal value) => + Math.Round(value, value >= 100 ? 0 : value >= 10 ? 1 : 2, MidpointRounding.AwayFromZero); + + private static string Significant(decimal value) => Round(value).ToString("0.##", CultureInfo.InvariantCulture); + + private static decimal ToDecimal(object value) => value switch + { + decimal d => d, + long l => l, + int i => i, + short s => s, + double d => (decimal)d, + float f => (decimal)f, + _ => throw new TemplateException($"The value {value} isn't a number, so it can't be formatted as a size or count."), + }; +} diff --git a/tests/Pgcheckup.Tests/Checks/TemplateRenderTests.cs b/tests/Pgcheckup.Tests/Checks/TemplateRenderTests.cs new file mode 100644 index 0000000..64369b7 --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/TemplateRenderTests.cs @@ -0,0 +1,125 @@ +using Pgcheckup.Checks; + +namespace Pgcheckup.Tests.Checks; + +public class TemplateRenderTests +{ + private static readonly Template SlotMessage = new( + [ + new TextPart("Slot "), + new ValuePart("subject", ValueFormat.Default), + new TextPart(" has been inactive"), + new SectionPart([new TextPart(" for "), new ValuePart("inactive_for", ValueFormat.Default)]), + new TextPart(" and is holding "), + new ValuePart("retained_wal", ValueFormat.Bytes), + new TextPart(" of WAL."), + ]); + + private static string Render(Template template, params (string Name, object? Value)[] values) => + template.Render(values.ToDictionary(v => v.Name, v => v.Value)); + + private static string RenderOne(object? value, ValueFormat format = ValueFormat.Default) => + Render(new Template([new ValuePart("v", format)]), ("v", value)); + + [Fact] + public void Renders_values_into_the_text() + { + Assert.Equal( + "Slot debezium has been inactive for 3 days and is holding 48 GB of WAL.", + Render(SlotMessage, ("subject", "debezium"), ("inactive_for", new TimeSpan(3, 4, 12, 33)), ("retained_wal", 51_539_607_552L))); + } + + [Fact] + public void Leaves_out_a_section_whose_value_is_null() + { + Assert.Equal( + "Slot debezium has been inactive and is holding 48 GB of WAL.", + Render(SlotMessage, ("subject", "debezium"), ("inactive_for", null), ("retained_wal", 51_539_607_552L))); + } + + [Fact] + public void Refuses_a_null_value_outside_a_section() + { + var error = Assert.Throws(() => + Render(SlotMessage, ("subject", null), ("inactive_for", null), ("retained_wal", 1L))); + + Assert.Contains("subject", error.Message); + } + + [Fact] + public void Refuses_a_value_the_query_did_not_return() + { + var error = Assert.Throws(() => Render(SlotMessage, ("subject", "debezium"))); + + Assert.Contains("inactive_for", error.Message); + } + + [Theory] + [InlineData(3, 4, 0, 0, "3 days")] + [InlineData(1, 23, 59, 0, "1 day")] + [InlineData(0, 4, 30, 0, "4 hours")] + [InlineData(0, 1, 0, 0, "1 hour")] + [InlineData(0, 0, 12, 59, "12 minutes")] + [InlineData(0, 0, 1, 0, "1 minute")] + [InlineData(0, 0, 0, 45, "45 seconds")] + [InlineData(0, 0, 0, 1, "1 second")] + [InlineData(0, 0, 0, 0, "0 seconds")] + public void Prints_intervals_in_their_largest_whole_unit(int days, int hours, int minutes, int seconds, string expected) + { + Assert.Equal(expected, RenderOne(new TimeSpan(days, hours, minutes, seconds))); + } + + [Theory] + [InlineData(4127L, "4127")] + [InlineData("orders", "orders")] + [InlineData(true, "on")] + [InlineData(false, "off")] + public void Prints_other_values_plainly(object value, string expected) + { + Assert.Equal(expected, RenderOne(value)); + } + + [Fact] + public void Prints_decimals_and_timestamps_without_culture() + { + Assert.Equal("0.95", RenderOne(0.95m)); + Assert.Equal("2026-09-25 14:03 UTC", RenderOne(new DateTime(2026, 9, 25, 14, 3, 59, DateTimeKind.Utc))); + } + + [Theory] + [InlineData(0L, "0 bytes")] + [InlineData(1L, "1 byte")] + [InlineData(512L, "512 bytes")] + [InlineData(1536L, "1.5 kB")] + [InlineData(1_610_612_736L, "1.5 GB")] + [InlineData(51_539_607_552L, "48 GB")] + [InlineData(1_073_741_823L, "1 GB")] + [InlineData(1_825_361_101L, "1.7 GB")] + [InlineData(13_421_772_800L, "12.5 GB")] + [InlineData(1_125_899_906_842_624L, "1 PB")] + public void Prints_bytes_like_postgres_sizes(long bytes, string expected) + { + Assert.Equal(expected, RenderOne(bytes, ValueFormat.Bytes)); + } + + [Fact] + public void Prints_numeric_bytes() + { + Assert.Equal("48 GB", RenderOne(51_539_607_552m, ValueFormat.Bytes)); + } + + [Theory] + [InlineData(48_213L, "48,213")] + [InlineData(999_999L, "999,999")] + [InlineData(1_000_000L, "1 million")] + [InlineData(48_000_000L, "48 million")] + [InlineData(123_456_789L, "123 million")] + [InlineData(1_610_000_000L, "1.61 billion")] + [InlineData(2_147_483_647L, "2.15 billion")] + [InlineData(999_999_999L, "1 billion")] + [InlineData(4_000_000_000_000L, "4 trillion")] + public void Prints_large_counts_in_words(long count, string expected) + { + Assert.Equal(expected, RenderOne(count, ValueFormat.Count)); + } +} From ee25934ff1e053a914dee6406047f687f14da24c Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 00:48:58 -0500 Subject: [PATCH 05/26] feat(checks): compile checks into the binary at build time --- .../CheckCompiler.cs | 304 ++++++++++++++++++ .../CheckGenerator.cs | 130 ++++++++ src/Pgcheckup.Checks.Generator/CheckSql.cs | 118 +++++++ src/Pgcheckup.Checks.Generator/Frontmatter.cs | 208 ++++++++++++ .../IsExternalInit.cs | 6 + .../Pgcheckup.Checks.Generator.csproj | 2 + .../SqlTokenizer.cs | 249 ++++++++++++++ .../TemplateParser.cs | 153 +++++++++ .../ThresholdValue.cs | 86 +++++ src/Pgcheckup/Pgcheckup.csproj | 10 + .../Checks/CheckCompilerTests.cs | 233 ++++++++++++++ .../Checks/CheckGeneratorTests.cs | 140 ++++++++ tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs | 123 +++++++ .../Checks/TemplateParserTests.cs | 55 ++++ tests/Pgcheckup.Tests/Checks/TemplateText.cs | 16 + .../Checks/ThresholdValueTests.cs | 65 ++++ tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj | 1 + 17 files changed, 1899 insertions(+) create mode 100644 src/Pgcheckup.Checks.Generator/CheckCompiler.cs create mode 100644 src/Pgcheckup.Checks.Generator/CheckGenerator.cs create mode 100644 src/Pgcheckup.Checks.Generator/CheckSql.cs create mode 100644 src/Pgcheckup.Checks.Generator/Frontmatter.cs create mode 100644 src/Pgcheckup.Checks.Generator/IsExternalInit.cs create mode 100644 src/Pgcheckup.Checks.Generator/SqlTokenizer.cs create mode 100644 src/Pgcheckup.Checks.Generator/TemplateParser.cs create mode 100644 src/Pgcheckup.Checks.Generator/ThresholdValue.cs create mode 100644 tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs create mode 100644 tests/Pgcheckup.Tests/Checks/CheckGeneratorTests.cs create mode 100644 tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs create mode 100644 tests/Pgcheckup.Tests/Checks/TemplateParserTests.cs create mode 100644 tests/Pgcheckup.Tests/Checks/TemplateText.cs create mode 100644 tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs diff --git a/src/Pgcheckup.Checks.Generator/CheckCompiler.cs b/src/Pgcheckup.Checks.Generator/CheckCompiler.cs new file mode 100644 index 0000000..6971704 --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/CheckCompiler.cs @@ -0,0 +1,304 @@ +using System.Collections.Generic; +using System.Globalization; +using System.Linq; +using System.Text.RegularExpressions; + +namespace Pgcheckup.Checks.Generator; + +public sealed class CheckFiles(string id, string? checkMd, string? checkSql, IReadOnlyCollection otherFiles) +{ + public string Id { get; } = id; + + public string? CheckMd { get; } = checkMd; + + public string? CheckSql { get; } = checkSql; + + // Paths relative to the check's folder, with forward slashes. + public IReadOnlyCollection OtherFiles { get; } = otherFiles; +} + +public sealed class CheckError(string file, int line, string message) +{ + public string File { get; } = file; + + public int Line { get; } = line; + + public string Message { get; } = message; + + public override string ToString() => $"{File}({Line}): {Message}"; +} + +public sealed class CompiledThreshold(string name, ThresholdValue value) +{ + public string Name { get; } = name; + + public ThresholdValue Value { get; } = value; +} + +public sealed class CompiledCheck +{ + public string Id { get; set; } = ""; + + public string Title { get; set; } = ""; + + public string Category { get; set; } = ""; + + public string Severity { get; set; } = ""; + + public int MinVersion { get; set; } + + public IReadOnlyList Privileges { get; set; } = []; + + public IReadOnlyList SkipOn { get; set; } = []; + + // In the order of their $n parameters. + public IReadOnlyList Thresholds { get; set; } = []; + + public string Sql { get; set; } = ""; + + public IReadOnlyList Message { get; set; } = []; + + public IReadOnlyList Fix { get; set; } = []; + + public string Note { get; set; } = ""; +} + +public sealed class CheckCompilation(CompiledCheck? check, IReadOnlyList errors) +{ + public CompiledCheck? Check { get; } = check; + + public IReadOnlyList Errors { get; } = errors; +} + +public static class CheckCompiler +{ + public const string CheckMd = "check.md"; + public const string CheckSqlFile = "check.sql"; + public const string FiresFixture = "fixtures/fires.sql"; + public const string HealthyFixture = "fixtures/healthy.sql"; + + public static readonly string[] Categories = ["ids", "cleanup", "wal", "capacity"]; + public static readonly string[] Severities = ["critical", "warning", "info"]; + public static readonly string[] Privileges = ["pg_monitor", "pg_read_all_settings", "pg_read_all_stats", "pg_stat_scan_tables"]; + public static readonly string[] Providers = ["rds", "aurora", "cloudsql", "azure", "supabase", "neon"]; + public static readonly string[] Sections = ["What breaks", "Fix", "Seen in"]; + + private static readonly string[] RequiredKeys = ["id", "title", "category", "severity", "min_version", "privileges", "message", "fix"]; + private static readonly string[] OptionalKeys = ["skip_on", "thresholds"]; + + private static readonly Regex KebabCase = new("^[a-z0-9]+(-[a-z0-9]+)*$", RegexOptions.CultureInvariant); + private static readonly Regex SnakeCase = new("^[a-z][a-z0-9_]*$", RegexOptions.CultureInvariant); + + public static CheckCompilation Compile(CheckFiles files) + { + var errors = new List(); + if (files.CheckMd == null) + { + errors.Add(new CheckError(CheckMd, 1, $"The check folder {files.Id} has no check.md.")); + return new CheckCompilation(null, errors); + } + + var document = Frontmatter.Parse(files.CheckMd); + errors.AddRange(document.Errors.Select(e => new CheckError(CheckMd, e.Line, e.Message))); + if (document.Entries.Count == 0) + { + return new CheckCompilation(null, errors); + } + + var check = new CompiledCheck { Note = document.Body }; + var entries = document.Entries.GroupBy(e => e.Key).ToDictionary(g => g.Key, g => g.First()); + + foreach (var entry in document.Entries.Where(e => !RequiredKeys.Contains(e.Key) && !OptionalKeys.Contains(e.Key))) + { + errors.Add(new CheckError(CheckMd, entry.Line, $"Unknown key {entry.Key}. Use {string.Join(", ", RequiredKeys.Concat(OptionalKeys))}.")); + } + + foreach (var missing in RequiredKeys.Where(k => !entries.ContainsKey(k))) + { + errors.Add(new CheckError(CheckMd, 1, $"The frontmatter has no {missing}.")); + } + + string? Scalar(string key) + { + if (!entries.TryGetValue(key, out var entry)) + { + return null; + } + + if (entry.Kind != EntryKind.Scalar || entry.Scalar.Length == 0) + { + errors.Add(new CheckError(CheckMd, entry.Line, $"{key} must be a single value.")); + return null; + } + + return entry.Scalar; + } + + void OneOf(string key, string? value, string[] allowed, System.Action set) + { + if (value == null) + { + return; + } + + if (allowed.Contains(value)) + { + set(value); + } + else + { + errors.Add(new CheckError(CheckMd, entries[key].Line, $"{key} is {value}. Use {string.Join(", ", allowed)}.")); + } + } + + IReadOnlyList ListOf(string key, string[] allowed) + { + if (!entries.TryGetValue(key, out var entry)) + { + return []; + } + + if (entry.Kind != EntryKind.List) + { + errors.Add(new CheckError(CheckMd, entry.Line, $"{key} must be a list, such as [{allowed[0]}], or [].")); + return []; + } + + foreach (var unknown in entry.Items.Where(i => !allowed.Contains(i))) + { + errors.Add(new CheckError(CheckMd, entry.Line, $"{key} includes {unknown}. Use {string.Join(", ", allowed)}.")); + } + + return entry.Items; + } + + IReadOnlyList Template(string key) + { + var text = Scalar(key); + if (text == null) + { + return []; + } + + var parts = TemplateParser.Parse(text, out var templateErrors); + errors.AddRange(templateErrors.Select(e => new CheckError(CheckMd, entries[key].ValueLine, $"{key}: {e}"))); + return parts; + } + + var id = Scalar("id"); + if (id != null && !KebabCase.IsMatch(id)) + { + errors.Add(new CheckError(CheckMd, entries["id"].Line, $"The id {id} must be kebab-case, such as replication-slot-inactive.")); + } + else if (id != null && id != files.Id) + { + errors.Add(new CheckError(CheckMd, entries["id"].Line, $"The id {id} must match the folder name, {files.Id}.")); + } + else if (id != null) + { + check.Id = id; + } + + check.Title = Scalar("title") ?? ""; + OneOf("category", Scalar("category"), Categories, v => check.Category = v); + OneOf("severity", Scalar("severity"), Severities, v => check.Severity = v); + + var minVersion = Scalar("min_version"); + if (minVersion != null) + { + if (int.TryParse(minVersion, NumberStyles.None, CultureInfo.InvariantCulture, out var major) && major >= 10) + { + check.MinVersion = major; + } + else + { + errors.Add(new CheckError(CheckMd, entries["min_version"].Line, $"min_version is {minVersion}. Use a Postgres major version, 10 or later, such as 14.")); + } + } + + check.Privileges = ListOf("privileges", Privileges); + check.SkipOn = ListOf("skip_on", Providers); + check.Message = Template("message"); + check.Fix = Template("fix"); + + var thresholds = new List(); + var thresholdNames = new List(); + if (entries.TryGetValue("thresholds", out var thresholdEntry)) + { + if (thresholdEntry.Kind != EntryKind.Map) + { + errors.Add(new CheckError(CheckMd, thresholdEntry.Line, "thresholds must be a list of `name: value` lines, indented under it.")); + } + + foreach (var (name, text, line) in thresholdEntry.Map) + { + thresholdNames.Add(name); + if (!SnakeCase.IsMatch(name)) + { + errors.Add(new CheckError(CheckMd, line, $"The threshold name {name} must be snake_case, such as min_retained_wal.")); + } + else if (!ThresholdValue.TryParse(text, out var value, out var error)) + { + errors.Add(new CheckError(CheckMd, line, $"{name}: {error}")); + } + else + { + thresholds.Add(new CompiledThreshold(name, value)); + } + } + } + + CheckBody(document.Body, BodyStartLine(files.CheckMd), errors); + + if (files.CheckSql == null) + { + errors.Add(new CheckError(CheckSqlFile, 1, $"The check folder {files.Id} has no check.sql.")); + } + else + { + // Invalid thresholds were reported above; naming them all here avoids a second error. + var sql = CheckSql.Compile(files.CheckSql, thresholdNames); + errors.AddRange(sql.Errors.Select(e => new CheckError(CheckSqlFile, e.Line, e.Message))); + check.Sql = sql.Sql; + check.Thresholds = sql.Parameters.SelectMany(p => thresholds.Where(t => t.Name == p)).ToList(); + } + + foreach (var fixture in new[] { FiresFixture, HealthyFixture }.Where(f => !files.OtherFiles.Contains(f))) + { + errors.Add(new CheckError(fixture, 1, $"The check folder {files.Id} has no {fixture}. Every check needs one that fires and one that stays quiet.")); + } + + return new CheckCompilation(errors.Count == 0 ? check : null, errors); + } + + private static void CheckBody(string body, int bodyStartLine, List errors) + { + var lines = body.Split('\n'); + foreach (var section in Sections) + { + var heading = System.Array.FindIndex(lines, l => l.TrimEnd() == "## " + section); + if (heading < 0) + { + errors.Add(new CheckError(CheckMd, bodyStartLine, $"check.md has no ## {section} section.")); + continue; + } + + if (section == "Seen in") + { + var content = lines.Skip(heading + 1).TakeWhile(l => !l.StartsWith("## ", System.StringComparison.Ordinal)); + if (!content.Any(l => l.Contains("https://"))) + { + errors.Add(new CheckError(CheckMd, bodyStartLine + heading, "## Seen in needs at least one https:// link to a public incident or the Postgres docs.")); + } + } + } + } + + private static int BodyStartLine(string markdown) + { + var lines = markdown.Replace("\r\n", "\n").Split('\n'); + var close = System.Array.FindIndex(lines, 1, l => l.TrimEnd() == "---"); + var first = System.Array.FindIndex(lines, close + 1, l => l.Trim().Length > 0); + return first < 0 ? close + 1 : first + 1; + } +} diff --git a/src/Pgcheckup.Checks.Generator/CheckGenerator.cs b/src/Pgcheckup.Checks.Generator/CheckGenerator.cs new file mode 100644 index 0000000..fc3d0e4 --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/CheckGenerator.cs @@ -0,0 +1,130 @@ +using System; +using System.Collections.Generic; +using System.Collections.Immutable; +using System.Globalization; +using System.Linq; +using System.Text; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.Text; + +namespace Pgcheckup.Checks.Generator; + +[Generator(LanguageNames.CSharp)] +public sealed class CheckGenerator : IIncrementalGenerator +{ + private const string ChecksDirProperty = "build_property.PgcheckupChecksDir"; + + private static readonly DiagnosticDescriptor InvalidCheck = new( + "PGC001", "Invalid check", "{0}", "pgcheckup", DiagnosticSeverity.Error, isEnabledByDefault: true); + + private static readonly DiagnosticDescriptor NoChecksDir = new( + "PGC002", "Checks folder not set", "{0}", "pgcheckup", DiagnosticSeverity.Error, isEnabledByDefault: true); + + public void Initialize(IncrementalGeneratorInitializationContext context) + { + var root = context.AnalyzerConfigOptionsProvider.Select((options, _) => + options.GlobalOptions.TryGetValue(ChecksDirProperty, out var dir) && !string.IsNullOrWhiteSpace(dir) + ? Normalize(dir).TrimEnd('/') + "/" + : null); + + var files = context.AdditionalTextsProvider + .Combine(root) + .Where(pair => pair.Right != null && Normalize(pair.Left.Path).StartsWith(pair.Right, StringComparison.OrdinalIgnoreCase)) + .Select((pair, cancellationToken) => + { + var relative = Normalize(pair.Left.Path).Substring(pair.Right!.Length); + var needsText = relative.EndsWith("/" + CheckCompiler.CheckMd, StringComparison.Ordinal) + || relative.EndsWith("/" + CheckCompiler.CheckSqlFile, StringComparison.Ordinal); + return new CheckFile(pair.Left.Path, relative, needsText ? pair.Left.GetText(cancellationToken)?.ToString() : null); + }) + .Collect(); + + context.RegisterSourceOutput(files.Combine(root), static (spc, input) => Emit(spc, input.Left, input.Right)); + } + + private static void Emit(SourceProductionContext context, ImmutableArray files, string? root) + { + if (root == null) + { + context.ReportDiagnostic(Diagnostic.Create(NoChecksDir, Location.None, "Set the PgcheckupChecksDir MSBuild property to the checks folder.")); + return; + } + + var compiled = new List(); + foreach (var folder in files.Where(f => f.Relative.Contains("/")).GroupBy(f => f.Relative.Substring(0, f.Relative.IndexOf('/'))).OrderBy(g => g.Key, StringComparer.Ordinal)) + { + string? Text(string name) => folder.FirstOrDefault(f => f.Relative == folder.Key + "/" + name)?.Text; + + var others = folder.Select(f => f.Relative.Substring(folder.Key.Length + 1)).ToList(); + var compilation = CheckCompiler.Compile(new CheckFiles(folder.Key, Text(CheckCompiler.CheckMd), Text(CheckCompiler.CheckSqlFile), others)); + foreach (var error in compilation.Errors) + { + var path = root + folder.Key + "/" + error.File; + var position = new LinePosition(Math.Max(0, error.Line - 1), 0); + var location = Location.Create(path, default, new LinePositionSpan(position, position)); + context.ReportDiagnostic(Diagnostic.Create(InvalidCheck, location, $"{folder.Key}: {error.Message}")); + } + + if (compilation.Check != null) + { + compiled.Add(compilation.Check); + } + } + + context.AddSource("CheckCatalog.g.cs", Source(compiled)); + } + + private static string Source(IReadOnlyList checks) + { + const string ns = "global::Pgcheckup.Checks."; + var code = new StringBuilder(); + code.AppendLine("// "); + code.AppendLine("#nullable enable"); + code.AppendLine("namespace Pgcheckup.Checks;"); + code.AppendLine(); + code.AppendLine("internal static partial class CheckCatalog"); + code.AppendLine("{"); + code.AppendLine($" public static global::System.Collections.Generic.IReadOnlyList<{ns}CheckDefinition> All {{ get; }} = new {ns}CheckDefinition[]"); + code.AppendLine(" {"); + foreach (var check in checks) + { + code.AppendLine($" new {ns}CheckDefinition("); + code.AppendLine($" Id: {Literal(check.Id)},"); + code.AppendLine($" Title: {Literal(check.Title)},"); + code.AppendLine($" Category: {Literal(check.Category)},"); + code.AppendLine($" Severity: {ns}Severity.{Pascal(check.Severity)},"); + code.AppendLine($" MinVersion: {check.MinVersion.ToString(CultureInfo.InvariantCulture)},"); + code.AppendLine($" Privileges: new string[] {{ {string.Join(", ", check.Privileges.Select(Literal))} }},"); + code.AppendLine($" SkipOn: new string[] {{ {string.Join(", ", check.SkipOn.Select(Literal))} }},"); + code.AppendLine($" Thresholds: new {ns}Threshold[] {{ {string.Join(", ", check.Thresholds.Select(t => $"new {ns}Threshold({Literal(t.Name)}, {ns}ThresholdKind.{t.Value.Kind}, {t.Value.Value.ToString(CultureInfo.InvariantCulture)}m, {Literal(t.Value.Text)})"))} }},"); + code.AppendLine($" Sql: {Literal(check.Sql)},"); + code.AppendLine($" Message: new {ns}Template({Parts(check.Message)}),"); + code.AppendLine($" Fix: new {ns}Template({Parts(check.Fix)}),"); + code.AppendLine($" Note: {Literal(check.Note)}),"); + } + + code.AppendLine(" };"); + code.AppendLine("}"); + return code.ToString(); + + static string Parts(IEnumerable parts) => + $"new {ns}TemplatePart[] {{ {string.Join(", ", parts.Select(Part))} }}"; + + static string Part(TemplatePart part) => part switch + { + TextPart t => $"new {ns}TextPart({Literal(t.Text)})", + ValuePart v => $"new {ns}ValuePart({Literal(v.Name)}, {ns}ValueFormat.{(v.Format == null ? "Default" : Pascal(v.Format))})", + SectionPart s => $"new {ns}SectionPart({Parts(s.Parts)})", + _ => throw new InvalidOperationException(), + }; + } + + private static string Literal(string value) => SymbolDisplay.FormatLiteral(value, quote: true); + + private static string Pascal(string value) => char.ToUpperInvariant(value[0]) + value.Substring(1); + + private static string Normalize(string path) => path.Replace('\\', '/'); + + private sealed record CheckFile(string Path, string Relative, string? Text); +} diff --git a/src/Pgcheckup.Checks.Generator/CheckSql.cs b/src/Pgcheckup.Checks.Generator/CheckSql.cs new file mode 100644 index 0000000..d508d7e --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/CheckSql.cs @@ -0,0 +1,118 @@ +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; + +namespace Pgcheckup.Checks.Generator; + +public sealed class SqlResult +{ + public SqlResult(string sql, IReadOnlyList parameters, IReadOnlyList errors) + { + Sql = sql; + Parameters = parameters; + Errors = errors; + } + + public string Sql { get; } + + public IReadOnlyList Parameters { get; } + + public IReadOnlyList Errors { get; } +} + +public static class CheckSql +{ + // A READ ONLY transaction blocks DDL, DML, nextval and row locks, but not these. They signal + // or reconfigure the server, take locks, consume transaction IDs or read server files. + private static readonly HashSet DeniedFunctions = new(StringComparer.Ordinal) + { + "nextval", "setval", "set_config", "txid_current", "pg_current_xact_id", + "pg_terminate_backend", "pg_cancel_backend", "pg_reload_conf", "pg_rotate_logfile", + "pg_switch_wal", "pg_promote", "pg_notify", "pg_export_snapshot", + "pg_logical_slot_get_changes", "pg_logical_slot_get_binary_changes", "pg_replication_slot_advance", + "pg_sync_replication_slots", "pg_log_standby_snapshot", "pg_log_backend_memory_contexts", + "pg_import_system_collations", "pg_create_restore_point", + "pg_backup_start", "pg_backup_stop", "pg_start_backup", "pg_stop_backup", + "pg_read_file", "pg_read_binary_file", + }; + + private static readonly string[] DeniedPrefixes = + [ + "pg_stat_reset", "pg_create_", "pg_drop_", "pg_copy_", "pg_advisory_", "pg_try_advisory_", + "pg_file_", "pg_wal_replay_", "pg_replication_origin_", "dblink", "lo_", + ]; + + public static SqlResult Compile(string sql, IReadOnlyCollection thresholds) + { + var errors = new List(); + var tokens = SqlTokenizer.Tokenize(sql, errors); + + var first = tokens.FirstOrDefault(); + if (first.Kind != TokenKind.Word || !(IsKeyword(first, "select") || IsKeyword(first, "with"))) + { + errors.Add(new SourceError(tokens.Count == 0 ? 1 : SqlTokenizer.LineOf(sql, first.Start), "check.sql must be one statement that starts with SELECT or WITH.")); + } + + var parameters = new List(); + var output = new StringBuilder(sql.Length); + var copied = 0; + for (var i = 0; i < tokens.Count; i++) + { + var token = tokens[i]; + switch (token.Kind) + { + case TokenKind.Semicolon when i != tokens.Count - 1: + errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), "check.sql must be one statement.")); + break; + + case TokenKind.Semicolon: + output.Append(sql, copied, token.Start - copied); + copied = token.Start + 1; + break; + + case TokenKind.Word or TokenKind.QuotedIdentifier when i + 1 < tokens.Count && tokens[i + 1].Text == "(": + var name = token.Text.ToLowerInvariant(); + if (DeniedFunctions.Contains(name) || DeniedPrefixes.Any(p => name.StartsWith(p, StringComparison.Ordinal))) + { + errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), $"check.sql calls {name}(), which has side effects that a READ ONLY transaction doesn't stop.")); + } + + break; + + case TokenKind.PositionalParameter: + errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), $"check.sql uses {token.Text}. Read thresholds as @name parameters.")); + break; + + case TokenKind.Parameter: + if (!thresholds.Contains(token.Text)) + { + errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), $"check.sql reads @{token.Text}, which isn't a threshold in check.md.")); + break; + } + + if (!parameters.Contains(token.Text)) + { + parameters.Add(token.Text); + } + + output.Append(sql, copied, token.Start - copied); + output.Append('$').Append(parameters.IndexOf(token.Text) + 1); + copied = token.Start + token.Length; + break; + } + } + + output.Append(sql, copied, sql.Length - copied); + + foreach (var unused in thresholds.Where(t => !parameters.Contains(t))) + { + errors.Add(new SourceError(1, $"Threshold {unused} is never read by check.sql.")); + } + + return new SqlResult(output.ToString(), parameters, errors); + } + + private static bool IsKeyword(Token token, string keyword) => + string.Equals(token.Text, keyword, StringComparison.OrdinalIgnoreCase); +} diff --git a/src/Pgcheckup.Checks.Generator/Frontmatter.cs b/src/Pgcheckup.Checks.Generator/Frontmatter.cs new file mode 100644 index 0000000..ffa5547 --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/Frontmatter.cs @@ -0,0 +1,208 @@ +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Text.RegularExpressions; + +namespace Pgcheckup.Checks.Generator; + +public enum EntryKind +{ + Scalar, + List, + Map, +} + +public sealed class FrontmatterEntry(string key, int line, EntryKind kind) +{ + public string Key { get; } = key; + + public int Line { get; } = line; + + // For block scalars, the line where the content starts. + public int ValueLine { get; set; } = line; + + public EntryKind Kind { get; } = kind; + + public string Scalar { get; set; } = ""; + + public List Items { get; } = []; + + public List<(string Key, string Value, int Line)> Map { get; } = []; +} + +public sealed class FrontmatterDocument +{ + public List Entries { get; } = []; + + public string Body { get; set; } = ""; + + public List Errors { get; } = []; +} + +// A strict subset of YAML: top-level `key: value`, `[a, b]` lists, one level of nested +// `name: value` maps, quoted scalars, and `|` or `>` block scalars. Anything else is an error +// with a line number, rather than something a full YAML parser would read differently. +public static class Frontmatter +{ + private static readonly Regex TopLevel = new(@"^([A-Za-z_][A-Za-z0-9_]*):(.*)$", RegexOptions.CultureInvariant); + private static readonly Regex Nested = new(@"^\s+([^:\s]+):(.*)$", RegexOptions.CultureInvariant); + + public static FrontmatterDocument Parse(string markdown) + { + var document = new FrontmatterDocument(); + var lines = markdown.Replace("\r\n", "\n").Split('\n'); + if (lines[0].TrimEnd() != "---") + { + document.Errors.Add(new SourceError(1, "check.md must start with frontmatter between --- lines.")); + return document; + } + + var close = System.Array.FindIndex(lines, 1, l => l.TrimEnd() == "---"); + if (close < 0) + { + document.Errors.Add(new SourceError(1, "The frontmatter's closing --- line is missing.")); + return document; + } + + document.Body = string.Join("\n", lines.Skip(close + 1)).Trim(); + + var i = 1; + while (i < close) + { + var line = lines[i]; + var lineNumber = i + 1; + i++; + + if (IsBlankOrComment(line)) + { + continue; + } + + var match = TopLevel.Match(line); + if (!match.Success) + { + document.Errors.Add(new SourceError(lineNumber, $"Expected `key: value`, not `{line.Trim()}`.")); + continue; + } + + var key = match.Groups[1].Value; + var rest = match.Groups[2].Value.Trim(); + if (document.Entries.Any(e => e.Key == key)) + { + document.Errors.Add(new SourceError(lineNumber, $"{key} appears more than once.")); + } + + FrontmatterEntry entry; + if (rest.Length == 0) + { + entry = new FrontmatterEntry(key, lineNumber, EntryKind.Map); + while (i < close && (IsBlankOrComment(lines[i]) || char.IsWhiteSpace(lines[i][0]))) + { + if (!IsBlankOrComment(lines[i])) + { + var nested = Nested.Match(lines[i]); + if (nested.Success) + { + entry.Map.Add((nested.Groups[1].Value, Unquote(nested.Groups[2].Value.Trim()), i + 1)); + } + else + { + document.Errors.Add(new SourceError(i + 1, $"Expected `name: value` under {key}.")); + } + } + + i++; + } + } + else if (rest is "|" or "|-" or "|+" or ">" or ">-" or ">+") + { + entry = new FrontmatterEntry(key, lineNumber, EntryKind.Scalar) { ValueLine = lineNumber + 1 }; + var block = new List(); + while (i < close && (lines[i].Trim().Length == 0 || char.IsWhiteSpace(lines[i][0]))) + { + block.Add(lines[i]); + i++; + } + + entry.Scalar = BlockScalar(block, folded: rest[0] == '>'); + } + else if (rest.StartsWith("[")) + { + entry = new FrontmatterEntry(key, lineNumber, EntryKind.List); + if (!rest.EndsWith("]")) + { + document.Errors.Add(new SourceError(lineNumber, $"The list for {key} must end with ] on the same line.")); + } + else + { + var inner = rest.Substring(1, rest.Length - 2); + entry.Items.AddRange(inner.Split(',').Select(s => Unquote(s.Trim())).Where(s => s.Length > 0)); + } + } + else + { + entry = new FrontmatterEntry(key, lineNumber, EntryKind.Scalar) { Scalar = Unquote(rest) }; + } + + document.Entries.Add(entry); + } + + return document; + } + + private static bool IsBlankOrComment(string line) + { + var trimmed = line.Trim(); + return trimmed.Length == 0 || trimmed[0] == '#'; + } + + private static string BlockScalar(List lines, bool folded) + { + var indent = lines.Where(l => l.Trim().Length > 0).Select(l => l.Length - l.TrimStart().Length).DefaultIfEmpty(0).Min(); + var content = lines.Select(l => l.Length >= indent ? l.Substring(indent).TrimEnd() : "").ToList(); + if (!folded) + { + return string.Join("\n", content).Trim('\n'); + } + + var text = new StringBuilder(); + foreach (var line in content) + { + if (line.Length == 0) + { + text.Append('\n'); + } + else + { + if (text.Length > 0 && text[text.Length - 1] != '\n') + { + text.Append(' '); + } + + text.Append(line); + } + } + + return text.ToString().Trim('\n'); + } + + private static string Unquote(string value) + { + if (value.Length > 0 && (value[0] == '"' || value[0] == '\'')) + { + var quote = value[0]; + var end = value.LastIndexOf(quote); + var after = value.Substring(end + 1).Trim(); + if (end > 0 && (after.Length == 0 || after[0] == '#')) + { + var inner = value.Substring(1, end - 1); + return quote == '"' + ? inner.Replace("\\\"", "\"").Replace("\\\\", "\\") + : inner.Replace("''", "'"); + } + } + + var comment = value.IndexOf(" #", System.StringComparison.Ordinal); + return comment >= 0 ? value.Substring(0, comment).TrimEnd() : value; + } +} diff --git a/src/Pgcheckup.Checks.Generator/IsExternalInit.cs b/src/Pgcheckup.Checks.Generator/IsExternalInit.cs new file mode 100644 index 0000000..70b0073 --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/IsExternalInit.cs @@ -0,0 +1,6 @@ +namespace System.Runtime.CompilerServices; + +// netstandard2.0 lacks this type, which records and init accessors need. +internal static class IsExternalInit +{ +} diff --git a/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj b/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj index c5d6335..b121d63 100644 --- a/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj +++ b/src/Pgcheckup.Checks.Generator/Pgcheckup.Checks.Generator.csproj @@ -4,6 +4,8 @@ disable true true + + $(NoWarn);RS2008 diff --git a/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs b/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs new file mode 100644 index 0000000..9b9f52b --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs @@ -0,0 +1,249 @@ +using System.Collections.Generic; + +namespace Pgcheckup.Checks.Generator; + +public sealed class SourceError +{ + public SourceError(int line, string message) + { + Line = line; + Message = message; + } + + public int Line { get; } + + public string Message { get; } + + public override string ToString() => $"line {Line}: {Message}"; +} + +public enum TokenKind +{ + Word, + QuotedIdentifier, + Parameter, + PositionalParameter, + Semicolon, + Other, +} + +public readonly struct Token +{ + public Token(TokenKind kind, int start, int length, string text) + { + Kind = kind; + Start = start; + Length = length; + Text = text; + } + + public TokenKind Kind { get; } + + public int Start { get; } + + public int Length { get; } + + public string Text { get; } +} + +// Just enough of Postgres's lexer to tell code from comments, string literals and quoted +// identifiers, so that checks on statements and function names can't be fooled by either. +public static class SqlTokenizer +{ + public static List Tokenize(string sql, List errors) + { + var tokens = new List(); + var i = 0; + while (i < sql.Length) + { + var c = sql[i]; + var next = i + 1 < sql.Length ? sql[i + 1] : '\0'; + var start = i; + + if (char.IsWhiteSpace(c)) + { + i++; + } + else if (c == '-' && next == '-') + { + while (i < sql.Length && sql[i] != '\n') + { + i++; + } + } + else if (c == '/' && next == '*') + { + i = SkipBlockComment(sql, i, errors); + } + else if (c == '\'') + { + var escapes = tokens.Count > 0 && IsEscapeStringPrefix(sql, tokens[tokens.Count - 1]); + i = SkipQuoted(sql, i, '\'', escapes, errors, "string literal"); + } + else if (c == '"') + { + i = SkipQuoted(sql, i, '"', false, errors, "quoted identifier"); + var content = sql.Substring(start + 1, System.Math.Max(0, i - start - 2)).Replace("\"\"", "\""); + tokens.Add(new Token(TokenKind.QuotedIdentifier, start, i - start, content)); + } + else if (c == '$' && char.IsDigit(next)) + { + i++; + while (i < sql.Length && char.IsDigit(sql[i])) + { + i++; + } + + tokens.Add(new Token(TokenKind.PositionalParameter, start, i - start, sql.Substring(start, i - start))); + } + else if (c == '$' && TryReadDollarTag(sql, i, out var tag)) + { + var end = sql.IndexOf(tag, i + tag.Length, System.StringComparison.Ordinal); + if (end < 0) + { + errors.Add(new SourceError(LineOf(sql, start), $"The {tag} string is never closed.")); + i = sql.Length; + } + else + { + i = end + tag.Length; + } + } + else if (IsIdentifierStart(c)) + { + while (i < sql.Length && IsIdentifierPart(sql[i])) + { + i++; + } + + tokens.Add(new Token(TokenKind.Word, start, i - start, sql.Substring(start, i - start))); + } + else if (c == '@' && IsIdentifierStart(next)) + { + i++; + while (i < sql.Length && IsIdentifierPart(sql[i]) && sql[i] != '$') + { + i++; + } + + tokens.Add(new Token(TokenKind.Parameter, start, i - start, sql.Substring(start + 1, i - start - 1))); + } + else if (c == ';') + { + i++; + tokens.Add(new Token(TokenKind.Semicolon, start, 1, ";")); + } + else + { + i++; + tokens.Add(new Token(TokenKind.Other, start, 1, c.ToString())); + } + } + + return tokens; + } + + private static bool IsEscapeStringPrefix(string sql, Token previous) => + previous.Kind == TokenKind.Word + && previous.Length == 1 + && (previous.Text == "E" || previous.Text == "e") + && previous.Start + 1 < sql.Length + && sql[previous.Start + 1] == '\''; + + private static int SkipBlockComment(string sql, int i, List errors) + { + var start = i; + var depth = 0; + while (i < sql.Length) + { + if (sql[i] == '/' && i + 1 < sql.Length && sql[i + 1] == '*') + { + depth++; + i += 2; + } + else if (sql[i] == '*' && i + 1 < sql.Length && sql[i + 1] == '/') + { + depth--; + i += 2; + if (depth == 0) + { + return i; + } + } + else + { + i++; + } + } + + errors.Add(new SourceError(LineOf(sql, start), "A /* comment is never closed.")); + return i; + } + + private static int SkipQuoted(string sql, int i, char quote, bool backslashEscapes, List errors, string what) + { + var start = i; + i++; + while (i < sql.Length) + { + if (backslashEscapes && sql[i] == '\\') + { + i += 2; + } + else if (sql[i] == quote && i + 1 < sql.Length && sql[i + 1] == quote) + { + i += 2; + } + else if (sql[i] == quote) + { + return i + 1; + } + else + { + i++; + } + } + + errors.Add(new SourceError(LineOf(sql, start), $"A {what} is never closed.")); + return sql.Length; + } + + private static bool TryReadDollarTag(string sql, int i, out string tag) + { + var j = i + 1; + if (j < sql.Length && IsIdentifierStart(sql[j])) + { + while (j < sql.Length && IsIdentifierPart(sql[j]) && sql[j] != '$') + { + j++; + } + } + + if (j < sql.Length && sql[j] == '$') + { + tag = sql.Substring(i, j - i + 1); + return true; + } + + tag = ""; + return false; + } + + private static bool IsIdentifierStart(char c) => char.IsLetter(c) || c == '_' || c > 127; + + private static bool IsIdentifierPart(char c) => IsIdentifierStart(c) || char.IsDigit(c) || c == '$'; + + internal static int LineOf(string text, int position) + { + var line = 1; + for (var i = 0; i < position && i < text.Length; i++) + { + if (text[i] == '\n') + { + line++; + } + } + + return line; + } +} diff --git a/src/Pgcheckup.Checks.Generator/TemplateParser.cs b/src/Pgcheckup.Checks.Generator/TemplateParser.cs new file mode 100644 index 0000000..02cb624 --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/TemplateParser.cs @@ -0,0 +1,153 @@ +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Text.RegularExpressions; + +namespace Pgcheckup.Checks.Generator; + +public abstract class TemplatePart +{ +} + +public sealed class TextPart(string text) : TemplatePart +{ + public string Text { get; } = text; +} + +public sealed class ValuePart(string name, string? format) : TemplatePart +{ + public string Name { get; } = name; + + public string? Format { get; } = format; +} + +public sealed class SectionPart(IReadOnlyList parts) : TemplatePart +{ + public IReadOnlyList Parts { get; } = parts; +} + +// Message and fix templates: {name} or {name:format} inserts a value, [ … ] is left out when a +// value inside it is NULL, and doubled braces or brackets stand for themselves. +public static class TemplateParser +{ + public static readonly string[] Formats = ["bytes", "count"]; + + private static readonly Regex Placeholder = new(@"^([a-z_][a-z0-9_]*)(?::([a-z]+))?$", RegexOptions.CultureInvariant); + + public static IReadOnlyList Parse(string template, out List errors) + { + errors = []; + if (template.Trim().Length == 0) + { + errors.Add("The template is empty."); + return []; + } + + var top = new List(); + List? section = null; + var text = new StringBuilder(); + + void FlushText() + { + if (text.Length > 0) + { + (section ?? top).Add(new TextPart(text.ToString())); + text.Clear(); + } + } + + var i = 0; + while (i < template.Length) + { + var c = template[i]; + var next = i + 1 < template.Length ? template[i + 1] : '\0'; + + if ((c == '{' || c == '}' || c == '[' || c == ']') && next == c) + { + text.Append(c); + i += 2; + continue; + } + + switch (c) + { + case '{': + var close = template.IndexOf('}', i + 1); + if (close < 0) + { + errors.Add($"A {{ at position {i + 1} is never closed."); + return top; + } + + var inner = template.Substring(i + 1, close - i - 1); + var match = Placeholder.Match(inner); + if (!match.Success) + { + errors.Add($"{{{inner}}} isn't a value. Write {{name}} or {{name:format}} with a lowercase column name."); + } + else if (match.Groups[2].Success && !Formats.Contains(match.Groups[2].Value)) + { + errors.Add($"{{{inner}}} uses the unknown format {match.Groups[2].Value}. Use {string.Join(" or ", Formats)}."); + } + else + { + FlushText(); + (section ?? top).Add(new ValuePart(match.Groups[1].Value, match.Groups[2].Success ? match.Groups[2].Value : null)); + } + + i = close + 1; + break; + + case '}': + errors.Add($"A }} at position {i + 1} has no matching {{. Write }}}} for a literal brace."); + i++; + break; + + case '[': + if (section != null) + { + errors.Add($"A [ at position {i + 1} is inside another [ … ] section."); + return top; + } + + FlushText(); + section = []; + i++; + break; + + case ']': + if (section == null) + { + errors.Add($"A ] at position {i + 1} has no matching [. Write ]] for a literal bracket."); + i++; + break; + } + + FlushText(); + if (!section.OfType().Any()) + { + errors.Add("A [ … ] section has no value in it, so it would never be left out."); + } + + top.Add(new SectionPart(section)); + section = null; + i++; + break; + + default: + text.Append(c); + i++; + break; + } + } + + if (section != null) + { + errors.Add("A [ … ] section is never closed."); + return top; + } + + FlushText(); + return top; + } +} diff --git a/src/Pgcheckup.Checks.Generator/ThresholdValue.cs b/src/Pgcheckup.Checks.Generator/ThresholdValue.cs new file mode 100644 index 0000000..c5fc75e --- /dev/null +++ b/src/Pgcheckup.Checks.Generator/ThresholdValue.cs @@ -0,0 +1,86 @@ +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Text.RegularExpressions; + +namespace Pgcheckup.Checks.Generator; + +public enum ThresholdKind +{ + Bytes, + Duration, + Integer, + Number, +} + +// Bytes and durations use Postgres's own units (as in postgresql.conf), so a threshold reads +// the same as the setting it is usually compared with. Durations are held in microseconds. +public readonly struct ThresholdValue +{ + private static readonly Regex Pattern = new(@"^(\d+(?:\.\d+)?)\s*([A-Za-z]*)$", RegexOptions.CultureInvariant); + + private static readonly Dictionary Units = new(StringComparer.Ordinal) + { + ["B"] = (ThresholdKind.Bytes, 1m), + ["kB"] = (ThresholdKind.Bytes, 1024m), + ["MB"] = (ThresholdKind.Bytes, 1024m * 1024), + ["GB"] = (ThresholdKind.Bytes, 1024m * 1024 * 1024), + ["TB"] = (ThresholdKind.Bytes, 1024m * 1024 * 1024 * 1024), + ["us"] = (ThresholdKind.Duration, 1m), + ["ms"] = (ThresholdKind.Duration, 1_000m), + ["s"] = (ThresholdKind.Duration, 1_000_000m), + ["min"] = (ThresholdKind.Duration, 60_000_000m), + ["h"] = (ThresholdKind.Duration, 3_600_000_000m), + ["d"] = (ThresholdKind.Duration, 86_400_000_000m), + }; + + public ThresholdValue(ThresholdKind kind, decimal value, string text) + { + Kind = kind; + Value = value; + Text = text; + } + + public ThresholdKind Kind { get; } + + public decimal Value { get; } + + public string Text { get; } + + public static bool TryParse(string text, out ThresholdValue value, out string error) + { + value = default; + if (string.IsNullOrWhiteSpace(text)) + { + error = "The threshold is empty."; + return false; + } + + var match = Pattern.Match(text.Trim()); + if (!match.Success) + { + error = $"'{text}' isn't a number with an optional unit (B, kB, MB, GB, TB, us, ms, s, min, h, d)."; + return false; + } + + var number = decimal.Parse(match.Groups[1].Value, NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture); + var unit = match.Groups[2].Value; + if (unit.Length == 0) + { + var kind = match.Groups[1].Value.Contains(".") ? ThresholdKind.Number : ThresholdKind.Integer; + value = new ThresholdValue(kind, number, text); + error = ""; + return true; + } + + if (!Units.TryGetValue(unit, out var known)) + { + error = $"'{text}' has an unknown unit. Use B, kB, MB, GB, TB, us, ms, s, min, h or d."; + return false; + } + + value = new ThresholdValue(known.Kind, Math.Round(number * known.Factor, MidpointRounding.AwayFromZero), text); + error = ""; + return true; + } +} diff --git a/src/Pgcheckup/Pgcheckup.csproj b/src/Pgcheckup/Pgcheckup.csproj index 1f8c0ab..3698197 100644 --- a/src/Pgcheckup/Pgcheckup.csproj +++ b/src/Pgcheckup/Pgcheckup.csproj @@ -5,9 +5,19 @@ Pgcheckup true false + $([System.IO.Path]::GetFullPath('$(MSBuildThisFileDirectory)../../checks/')) + + + + + + + + + diff --git a/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs b/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs new file mode 100644 index 0000000..8ec060b --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs @@ -0,0 +1,233 @@ +using Pgcheckup.Checks.Generator; + +namespace Pgcheckup.Tests.Checks; + +public class CheckCompilerTests +{ + private const string ValidFrontmatter = """ + id: sample-check + title: Sample check + category: wal + severity: warning + min_version: 14 + privileges: [pg_monitor] + thresholds: + min_size: 1GB + min_age: 1h + message: >- + Thing {subject} is [{age} ]old + and big. + fix: | + Do this: + SELECT 1; + """; + + private const string ValidBody = """ + ## What breaks + + Things. + + ## Fix + + Fix it. + + ## Seen in + + - [Replication slots](https://www.postgresql.org/docs/current/warm-standby.html) + """; + + private const string ValidSql = "SELECT 'x' AS subject WHERE 1 >= @min_age AND 2 >= @min_size"; + + private static readonly string[] BothFixtures = ["fixtures/fires.sql", "fixtures/healthy.sql"]; + + private static string Markdown(string frontmatter = ValidFrontmatter, string body = ValidBody) => + $"---\n{frontmatter}\n---\n\n{body}\n"; + + private static CheckCompilation Compile( + string? markdown = null, string? sql = ValidSql, string[]? otherFiles = null, string id = "sample-check") => + CheckCompiler.Compile(new CheckFiles(id, markdown ?? Markdown(), sql, otherFiles ?? BothFixtures)); + + private static CheckError SingleError(CheckCompilation compilation) + { + Assert.Null(compilation.Check); + return Assert.Single(compilation.Errors); + } + + [Fact] + public void Compiles_a_valid_check() + { + var compilation = Compile(); + + Assert.Empty(compilation.Errors); + var check = Assert.IsType(compilation.Check); + Assert.Equal("sample-check", check.Id); + Assert.Equal("Sample check", check.Title); + Assert.Equal("wal", check.Category); + Assert.Equal("warning", check.Severity); + Assert.Equal(14, check.MinVersion); + Assert.Equal(["pg_monitor"], check.Privileges); + Assert.Empty(check.SkipOn); + Assert.Equal("SELECT 'x' AS subject WHERE 1 >= $1 AND 2 >= $2", check.Sql); + Assert.Equal("'Thing '' is '[' ']'old and big.'", TemplateText.Describe(check.Message)); + Assert.Equal("'Do this:\nSELECT 1;'", TemplateText.Describe(check.Fix)); + Assert.StartsWith("## What breaks", check.Note); + } + + [Fact] + public void Orders_thresholds_by_their_parameter_position() + { + var check = Compile().Check!; + + Assert.Collection( + check.Thresholds, + t => + { + Assert.Equal("min_age", t.Name); + Assert.Equal(ThresholdKind.Duration, t.Value.Kind); + Assert.Equal(3_600_000_000m, t.Value.Value); + }, + t => + { + Assert.Equal("min_size", t.Name); + Assert.Equal(ThresholdKind.Bytes, t.Value.Kind); + Assert.Equal(1_073_741_824m, t.Value.Value); + }); + } + + [Fact] + public void Reads_quoted_scalars_and_ignores_comments() + { + var frontmatter = ValidFrontmatter + .Replace("title: Sample check", "# A comment line\ntitle: \"Sample: \\\"check\\\"\"") + .Replace("category: wal", "category: 'wal' # trailing comment"); + + var check = Compile(Markdown(frontmatter)).Check!; + + Assert.Equal("Sample: \"check\"", check.Title); + Assert.Equal("wal", check.Category); + } + + [Fact] + public void Requires_frontmatter() + { + var error = SingleError(Compile(ValidBody)); + + Assert.Equal("check.md", error.File); + Assert.Equal(1, error.Line); + Assert.Contains("---", error.Message); + } + + [Fact] + public void Reports_an_unknown_key_on_its_line() + { + var error = SingleError(Compile(Markdown(ValidFrontmatter + "\ncolour: blue"))); + + Assert.Equal(17, error.Line); + Assert.Contains("colour", error.Message); + } + + [Theory] + [InlineData("id")] + [InlineData("title")] + [InlineData("category")] + [InlineData("severity")] + [InlineData("min_version")] + [InlineData("privileges")] + public void Reports_a_missing_required_key(string key) + { + var frontmatter = string.Join("\n", ValidFrontmatter.Split('\n').Where(l => !l.StartsWith(key + ":", StringComparison.Ordinal))); + + Assert.Contains(key, SingleError(Compile(Markdown(frontmatter))).Message); + } + + [Fact] + public void Reports_a_duplicate_key() + { + Assert.Contains("title", SingleError(Compile(Markdown(ValidFrontmatter + "\ntitle: Again"))).Message); + } + + [Fact] + public void Requires_the_id_to_match_the_folder() + { + Assert.Contains("other-check", SingleError(Compile(id: "other-check")).Message); + } + + [Theory] + [InlineData("id: sample-check", "id: Sample_Check", "Sample_Check")] + [InlineData("category: wal", "category: disks", "disks")] + [InlineData("severity: warning", "severity: high", "high")] + [InlineData("min_version: 14", "min_version: 9.6", "9.6")] + [InlineData("min_version: 14", "min_version: 9", "9")] + [InlineData("privileges: [pg_monitor]", "privileges: [superuser]", "superuser")] + [InlineData("privileges: [pg_monitor]", "privileges: [pg_monitor]\nskip_on: [heroku]", "heroku")] + [InlineData("min_size: 1GB", "min_size: 1 gigabyte", "1 gigabyte")] + [InlineData("min_size: 1GB", "Min_Size: 1GB", "Min_Size")] + [InlineData("title: Sample check", "title: [a, b]", "title")] + public void Rejects_invalid_values(string valid, string invalid, string named) + { + var markdown = Markdown(ValidFrontmatter.Replace(valid, invalid)); + + Assert.Contains(Compile(markdown).Errors, e => e.File == "check.md" && e.Message.Contains(named)); + } + + [Fact] + public void Reports_a_bad_threshold_on_its_line() + { + var error = Assert.Single(Compile(Markdown(ValidFrontmatter.Replace("min_age: 1h", "min_age: 1 hour"))).Errors); + + Assert.Equal(10, error.Line); + } + + [Fact] + public void Reports_template_errors_on_the_template_line() + { + var error = SingleError(Compile(Markdown(ValidFrontmatter.Replace("{subject}", "{subject:gb}")))); + + Assert.Equal(12, error.Line); + Assert.Contains("gb", error.Message); + } + + [Theory] + [InlineData("## What breaks", "What breaks")] + [InlineData("## Fix", "Fix")] + [InlineData("## Seen in", "Seen in")] + public void Requires_every_section(string heading, string named) + { + var body = ValidBody.Replace(heading, "## Something else"); + + Assert.Contains(named, SingleError(Compile(Markdown(body: body))).Message); + } + + [Fact] + public void Requires_a_link_under_seen_in() + { + var body = ValidBody.Replace("- [Replication slots](https://www.postgresql.org/docs/current/warm-standby.html)", "- A talk I remember"); + + Assert.Contains("link", SingleError(Compile(Markdown(body: body))).Message); + } + + [Theory] + [InlineData("fixtures/fires.sql")] + [InlineData("fixtures/healthy.sql")] + public void Requires_both_fixtures(string fixture) + { + var error = SingleError(Compile(otherFiles: BothFixtures.Where(f => f != fixture).ToArray())); + + Assert.Equal(fixture, error.File); + } + + [Fact] + public void Requires_check_sql() + { + Assert.Equal("check.sql", SingleError(Compile(sql: null)).File); + } + + [Fact] + public void Reports_sql_errors_against_check_sql() + { + var error = SingleError(Compile(sql: ValidSql + "\n AND pg_terminate_backend(1) IS NULL")); + + Assert.Equal("check.sql", error.File); + Assert.Equal(2, error.Line); + } +} diff --git a/tests/Pgcheckup.Tests/Checks/CheckGeneratorTests.cs b/tests/Pgcheckup.Tests/Checks/CheckGeneratorTests.cs new file mode 100644 index 0000000..2aa5490 --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/CheckGeneratorTests.cs @@ -0,0 +1,140 @@ +using System.Collections.Immutable; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.Diagnostics; +using Microsoft.CodeAnalysis.Text; +using Pgcheckup.Checks.Generator; + +namespace Pgcheckup.Tests.Checks; + +public class CheckGeneratorTests +{ + private const string Root = "/repo/checks/"; + + private const string ValidCheckMd = """ + --- + id: good-check + title: Good check + category: wal + severity: warning + min_version: 14 + privileges: [pg_monitor] + thresholds: + min_size: 1GB + message: Slot {subject}[ for {age}] holds {size:bytes} and "quotes". + fix: | + SELECT pg_drop_replication_slot({slot_literal}); + SELECT ARRAY[[1]]; + --- + + ## What breaks + + Disks fill. + + ## Fix + + Drop it. + + ## Seen in + + - [Docs](https://www.postgresql.org/docs/current/warm-standby.html) + """; + + private static GeneratorDriverRunResult Run(params (string Path, string Text)[] files) + { + var compilation = CSharpCompilation.Create( + "Generated", + references: TrustedPlatformReferences(), + options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + + var driver = CSharpGeneratorDriver.Create( + [new CheckGenerator().AsSourceGenerator()], + files.Select(f => (AdditionalText)new InMemoryText(f.Path, f.Text)), + optionsProvider: new BuildProperties(new() { ["build_property.PgcheckupChecksDir"] = Root })); + + driver = (CSharpGeneratorDriver)driver.RunGeneratorsAndUpdateCompilation(compilation, out var output, out _); + + // The generated catalog must compile against the real runtime types. + Assert.Empty(output.GetDiagnostics().Where(d => d.Severity == DiagnosticSeverity.Error)); + return driver.GetRunResult(); + } + + private static IEnumerable TrustedPlatformReferences() => + ((string)AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")!) + .Split(Path.PathSeparator) + .Append(typeof(Pgcheckup.Checks.CheckDefinition).Assembly.Location) + .Distinct(StringComparer.OrdinalIgnoreCase) + .Select(p => MetadataReference.CreateFromFile(p)); + + private static (string, string)[] GoodCheck(string folder = "good-check") => + [ + ($"{Root}{folder}/check.md", ValidCheckMd), + ($"{Root}{folder}/check.sql", "SELECT 'x' AS subject WHERE 1 > @min_size"), + ($"{Root}{folder}/fixtures/fires.sql", ""), + ($"{Root}{folder}/fixtures/healthy.sql", ""), + ]; + + [Fact] + public void Emits_a_catalog_that_compiles_for_valid_checks() + { + var result = Run(GoodCheck()); + + Assert.Empty(result.Diagnostics); + var source = Assert.Single(result.GeneratedTrees).ToString(); + Assert.Contains("\"good-check\"", source); + } + + [Fact] + public void Reports_an_invalid_check_as_an_error_on_its_file_and_line() + { + var files = GoodCheck(); + files[0].Item2 = ValidCheckMd.Replace("title: Good check", "title: Good check\ncolour: blue"); + + var diagnostic = Assert.Single(Run(files).Diagnostics); + + Assert.Equal(DiagnosticSeverity.Error, diagnostic.Severity); + Assert.Equal("PGC001", diagnostic.Id); + Assert.Contains("colour", diagnostic.GetMessage()); + var span = diagnostic.Location.GetLineSpan(); + Assert.Equal($"{Root}good-check/check.md", span.Path); + Assert.Equal(4, span.StartLinePosition.Line + 1); + } + + [Fact] + public void Reports_a_check_folder_without_check_md() + { + var diagnostic = Assert.Single(Run(($"{Root}lost-check/check.sql", "SELECT 1")).Diagnostics); + + Assert.Equal(DiagnosticSeverity.Error, diagnostic.Severity); + Assert.Contains("lost-check", diagnostic.GetMessage()); + } + + [Fact] + public void Ignores_files_outside_check_folders() + { + var result = Run(("/repo/docs/check.md", "not a check"), ($"{Root}README.md", "About checks")); + + Assert.Empty(result.Diagnostics); + } + + private sealed class InMemoryText(string path, string text) : AdditionalText + { + public override string Path { get; } = path; + + public override SourceText GetText(CancellationToken cancellationToken = default) => SourceText.From(text); + } + + private sealed class BuildProperties(Dictionary global) : AnalyzerConfigOptionsProvider + { + public override AnalyzerConfigOptions GlobalOptions { get; } = new Options(global); + + public override AnalyzerConfigOptions GetOptions(SyntaxTree tree) => new Options([]); + + public override AnalyzerConfigOptions GetOptions(AdditionalText textFile) => new Options([]); + + private sealed class Options(Dictionary values) : AnalyzerConfigOptions + { + public override bool TryGetValue(string key, out string value) => values.TryGetValue(key, out value!); + } + } +} diff --git a/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs b/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs new file mode 100644 index 0000000..368552b --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs @@ -0,0 +1,123 @@ +using Pgcheckup.Checks.Generator; + +namespace Pgcheckup.Tests.Checks; + +public class CheckSqlTests +{ + private static SqlResult Compile(string sql, params string[] thresholds) => CheckSql.Compile(sql, thresholds); + + [Fact] + public void Rewrites_named_thresholds_to_positional_parameters_in_order_of_first_use() + { + var result = Compile("SELECT 1 WHERE a >= @min_x AND b < @max_y OR c = @min_x", "max_y", "min_x"); + + Assert.Empty(result.Errors); + Assert.Equal("SELECT 1 WHERE a >= $1 AND b < $2 OR c = $1", result.Sql); + Assert.Equal(["min_x", "max_y"], result.Parameters); + } + + [Fact] + public void Leaves_at_signs_alone_inside_literals_comments_and_operators() + { + const string sql = """ + SELECT '@x', E'\'@x', $q$ @x $q$, "@x", a @> b -- @x + /* @x /* nested @x */ @x */ + WHERE n >= @x + """; + + var result = Compile(sql, "x"); + + Assert.Empty(result.Errors); + Assert.Equal(sql.Replace("n >= @x", "n >= $1"), result.Sql); + } + + [Fact] + public void Reports_an_unknown_threshold_with_its_line() + { + var result = Compile("SELECT 1\nWHERE a > @nope", "x"); + + Assert.Contains(result.Errors, e => e.Line == 2 && e.Message.Contains("@nope")); + } + + [Fact] + public void Reports_a_threshold_the_sql_never_reads() + { + var result = Compile("SELECT 1", "unused_limit"); + + Assert.Contains(result.Errors, e => e.Message.Contains("unused_limit")); + } + + [Theory] + [InlineData("SELECT 1")] + [InlineData("select 1")] + [InlineData("-- leading comment\nWITH x AS (SELECT 1) SELECT * FROM x")] + [InlineData("/* c */ SELECT 1;")] + public void Accepts_one_select_or_with_statement(string sql) + { + Assert.Empty(Compile(sql).Errors); + } + + [Fact] + public void Drops_a_trailing_semicolon() + { + Assert.Equal("SELECT 1\n", Compile("SELECT 1;\n").Sql); + } + + [Theory] + [InlineData("INSERT INTO t VALUES (1)")] + [InlineData("SET statement_timeout = 0")] + [InlineData("(SELECT 1)")] + [InlineData("")] + [InlineData("-- only a comment")] + public void Rejects_anything_but_a_select_or_with_statement(string sql) + { + Assert.Contains(Compile(sql).Errors, e => e.Message.Contains("SELECT or WITH")); + } + + [Theory] + [InlineData("SELECT 1; SELECT 2")] + [InlineData("SELECT 1;;")] + [InlineData("SELECT 1; -- trailing\nDELETE FROM t")] + public void Rejects_more_than_one_statement(string sql) + { + Assert.Contains(Compile(sql).Errors, e => e.Message.Contains("one statement")); + } + + [Theory] + [InlineData("SELECT pg_terminate_backend(pid) FROM pg_stat_activity")] + [InlineData("SELECT PG_CANCEL_BACKEND(1)")] + [InlineData("SELECT pg_catalog.pg_reload_conf()")] + [InlineData("SELECT \"pg_terminate_backend\"(1)")] + [InlineData("SELECT nextval('s')")] + [InlineData("SELECT set_config('x', 'y', true)")] + [InlineData("SELECT txid_current()")] + [InlineData("SELECT pg_current_xact_id()")] + [InlineData("SELECT pg_advisory_lock(1)")] + [InlineData("SELECT pg_try_advisory_xact_lock(1)")] + [InlineData("SELECT pg_stat_reset_shared('wal')")] + [InlineData("SELECT pg_create_physical_replication_slot('s')")] + [InlineData("SELECT pg_drop_replication_slot('s')")] + [InlineData("SELECT pg_logical_slot_get_changes('s', NULL, NULL)")] + [InlineData("SELECT dblink('x', 'y')")] + [InlineData("SELECT lo_import('/etc/passwd')")] + [InlineData("SELECT pg_read_file('postgresql.conf')")] + [InlineData("SELECT pg_switch_wal()")] + [InlineData("SELECT pg_notify('c', 'x')")] + public void Rejects_functions_with_side_effects(string sql) + { + Assert.Contains(Compile(sql).Errors, e => e.Line == 1 && e.Message.Contains("side effects")); + } + + [Theory] + [InlineData("SELECT 'pg_terminate_backend'")] + [InlineData("SELECT 1 -- pg_terminate_backend(pid)")] + [InlineData("SELECT pg_logical_slot_peek_changes('s', NULL, NULL)")] + [InlineData("SELECT txid_current_if_assigned()")] + [InlineData("SELECT pg_current_xact_id_if_assigned()")] + [InlineData("SELECT pg_last_wal_replay_lsn()")] + [InlineData("SELECT pg_ls_waldir()")] + public void Allows_read_only_functions_and_mentions_in_literals(string sql) + { + Assert.Empty(Compile(sql).Errors); + } +} diff --git a/tests/Pgcheckup.Tests/Checks/TemplateParserTests.cs b/tests/Pgcheckup.Tests/Checks/TemplateParserTests.cs new file mode 100644 index 0000000..44fdc89 --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/TemplateParserTests.cs @@ -0,0 +1,55 @@ +using Pgcheckup.Checks.Generator; + +namespace Pgcheckup.Tests.Checks; + +public class TemplateParserTests +{ + private static string Parse(string template) + { + var parts = TemplateParser.Parse(template, out var errors); + Assert.Empty(errors); + return TemplateText.Describe(parts); + } + + private static IReadOnlyList Errors(string template) + { + TemplateParser.Parse(template, out var errors); + return errors; + } + + [Fact] + public void Splits_text_values_and_optional_sections() + { + Assert.Equal( + "'Slot '' has been inactive'[' for ']' and holds ''.'", + Parse("Slot {subject} has been inactive[ for {inactive_for}] and holds {retained_wal:bytes}.")); + } + + [Fact] + public void Doubled_braces_and_brackets_are_literal() + { + Assert.Equal("'SELECT ARRAY[1] {x}'", Parse("SELECT ARRAY[[1]] {{x}}")); + } + + [Fact] + public void Accepts_the_count_format() + { + Assert.Equal("' IDs'", Parse("{age:count} IDs")); + } + + [Theory] + [InlineData("Slot {subject", "never closed")] + [InlineData("Slot {Subject}", "Subject")] + [InlineData("Slot {}", "{}")] + [InlineData("{size:megabytes}", "megabytes")] + [InlineData("a } b", "}")] + [InlineData("[ for {x}", "never closed")] + [InlineData("a ] b", "]")] + [InlineData("[a [b {x}]]", "inside")] + [InlineData("a[ no values ]", "no value")] + [InlineData("", "empty")] + public void Rejects_malformed_templates(string template, string expected) + { + Assert.Contains(Errors(template), e => e.Contains(expected)); + } +} diff --git a/tests/Pgcheckup.Tests/Checks/TemplateText.cs b/tests/Pgcheckup.Tests/Checks/TemplateText.cs new file mode 100644 index 0000000..4bf449c --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/TemplateText.cs @@ -0,0 +1,16 @@ +using Pgcheckup.Checks.Generator; + +namespace Pgcheckup.Tests.Checks; + +// Writes template parts back in a compact form so expectations can be written by hand. +internal static class TemplateText +{ + public static string Describe(IEnumerable parts) => string.Concat(parts.Select(p => p switch + { + TextPart t => $"'{t.Text}'", + ValuePart { Format: null } v => $"<{v.Name}>", + ValuePart v => $"<{v.Name}:{v.Format}>", + SectionPart s => $"[{Describe(s.Parts)}]", + _ => throw new InvalidOperationException(), + })); +} diff --git a/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs b/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs new file mode 100644 index 0000000..fe0c9ff --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs @@ -0,0 +1,65 @@ +using Pgcheckup.Checks.Generator; + +namespace Pgcheckup.Tests.Checks; + +public class ThresholdValueTests +{ + [Theory] + [InlineData("0B", 0)] + [InlineData("1GB", 1073741824)] + [InlineData("512 MB", 536870912)] + [InlineData("1.5kB", 1536)] + [InlineData("2TB", 2199023255552)] + public void Parses_bytes_with_postgres_units(string text, long bytes) + { + Assert.True(ThresholdValue.TryParse(text, out var value, out _)); + Assert.Equal(ThresholdKind.Bytes, value.Kind); + Assert.Equal(bytes, value.Value); + } + + [Theory] + [InlineData("0s", 0)] + [InlineData("250ms", 250_000)] + [InlineData("5s", 5_000_000)] + [InlineData("30min", 1_800_000_000)] + [InlineData("1h", 3_600_000_000)] + [InlineData("2 d", 172_800_000_000)] + [InlineData("100us", 100)] + public void Parses_durations_to_microseconds(string text, long microseconds) + { + Assert.True(ThresholdValue.TryParse(text, out var value, out _)); + Assert.Equal(ThresholdKind.Duration, value.Kind); + Assert.Equal(microseconds, value.Value); + } + + [Fact] + public void Parses_a_plain_integer() + { + Assert.True(ThresholdValue.TryParse("1500000000", out var value, out _)); + Assert.Equal(ThresholdKind.Integer, value.Kind); + Assert.Equal(1_500_000_000m, value.Value); + } + + [Fact] + public void Parses_a_decimal_number() + { + Assert.True(ThresholdValue.TryParse("0.9", out var value, out _)); + Assert.Equal(ThresholdKind.Number, value.Kind); + Assert.Equal(0.9m, value.Value); + } + + [Theory] + [InlineData("")] + [InlineData("1gb")] + [InlineData("1 GiB")] + [InlineData("-1h")] + [InlineData("1e9")] + [InlineData("90%")] + [InlineData("h")] + [InlineData("1.5.1s")] + public void Rejects_values_postgres_would_not_accept(string text) + { + Assert.False(ThresholdValue.TryParse(text, out _, out var error)); + Assert.Contains(text.Length == 0 ? "empty" : text, error); + } +} diff --git a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj index c4674b1..93772b8 100644 --- a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj +++ b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj @@ -4,6 +4,7 @@ false + From 781196d65a96d78fad663e17cbfc28b46c519df4 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:00:01 -0500 Subject: [PATCH 06/26] feat(engine): run every query in its own guarded read-only transaction --- Directory.Build.props | 5 + src/Pgcheckup/Engine/ReadOnlySession.cs | 95 ++++++++++++++ .../Engine/ReadOnlySessionTests.cs | 119 ++++++++++++++++++ .../Postgres/PostgresServer.cs | 63 ++++++++++ 4 files changed, 282 insertions(+) create mode 100644 src/Pgcheckup/Engine/ReadOnlySession.cs create mode 100644 tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs create mode 100644 tests/Pgcheckup.Tests/Postgres/PostgresServer.cs diff --git a/Directory.Build.props b/Directory.Build.props index 8f38ccc..cee89fc 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -9,4 +9,9 @@ 0.0.0 false + + + + diff --git a/src/Pgcheckup/Engine/ReadOnlySession.cs b/src/Pgcheckup/Engine/ReadOnlySession.cs new file mode 100644 index 0000000..6be0aa2 --- /dev/null +++ b/src/Pgcheckup/Engine/ReadOnlySession.cs @@ -0,0 +1,95 @@ +using Npgsql; + +namespace Pgcheckup.Engine; + +// The only way pgcheckup talks to Postgres. Every query runs in its own READ ONLY transaction +// with transaction-local timeouts, then rolls back. Nothing is set for the session: behind a +// transaction pooler, a session setting would reach the application's next transaction. +public sealed class ReadOnlySession : IAsyncDisposable +{ + private static readonly string[] Guards = + [ + "BEGIN TRANSACTION READ ONLY", + "SET LOCAL statement_timeout = '5s'", + + // Stops a scan from queueing behind a migration's lock and blocking the traffic behind it. + "SET LOCAL lock_timeout = '1s'", + ]; + + private readonly NpgsqlDataSource dataSource; + private readonly NpgsqlConnection connection; + + private ReadOnlySession(NpgsqlDataSource dataSource, NpgsqlConnection connection) + { + this.dataSource = dataSource; + this.connection = connection; + } + + public static async Task OpenAsync(NpgsqlConnectionStringBuilder settings, CancellationToken cancellationToken) + { + var builder = new NpgsqlSlimDataSourceBuilder(settings.ConnectionString); + builder.ConnectionStringBuilder.ApplicationName = "pgcheckup"; + builder.ConnectionStringBuilder.Pooling = false; + builder.EnableTransportSecurity(); + + // Loading types would run a query outside the guarded transaction; the built-in types suffice. + builder.ConfigureTypeLoading(options => options.EnableTypeLoading(false)); + + var dataSource = builder.Build(); + try + { + var connection = await dataSource.OpenConnectionAsync(cancellationToken); + return new ReadOnlySession(dataSource, connection); + } + catch + { + await dataSource.DisposeAsync(); + throw; + } + } + + public async Task>> QueryAsync( + string sql, IReadOnlyList parameters, CancellationToken cancellationToken) + { + foreach (var guard in Guards) + { + await using var command = new NpgsqlCommand(guard, connection); + await command.ExecuteNonQueryAsync(cancellationToken); + } + + try + { + await using var query = new NpgsqlCommand(sql, connection); + foreach (var parameter in parameters) + { + query.Parameters.Add(new NpgsqlParameter { Value = parameter }); + } + + var rows = new List>(); + await using var reader = await query.ExecuteReaderAsync(cancellationToken); + while (await reader.ReadAsync(cancellationToken)) + { + var row = new Dictionary(reader.FieldCount, StringComparer.Ordinal); + for (var i = 0; i < reader.FieldCount; i++) + { + row[reader.GetName(i)] = await reader.IsDBNullAsync(i, cancellationToken) ? null : reader.GetValue(i); + } + + rows.Add(row); + } + + return rows; + } + finally + { + await using var rollback = new NpgsqlCommand("ROLLBACK", connection); + await rollback.ExecuteNonQueryAsync(CancellationToken.None); + } + } + + public async ValueTask DisposeAsync() + { + await connection.DisposeAsync(); + await dataSource.DisposeAsync(); + } +} diff --git a/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs b/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs new file mode 100644 index 0000000..a1262ef --- /dev/null +++ b/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs @@ -0,0 +1,119 @@ +using Npgsql; +using Pgcheckup.Engine; +using Pgcheckup.Tests.Postgres; + +namespace Pgcheckup.Tests.Engine; + +public class ReadOnlySessionTests(PostgresServerFixture postgres) : IClassFixture +{ + private static CancellationToken Cancel => TestContext.Current.CancellationToken; + + private async Task>> QueryAsync(string sql, params object[] parameters) + { + await using var session = await ReadOnlySession.OpenAsync(postgres.Server.Checkup, Cancel); + return await session.QueryAsync(sql, parameters, Cancel); + } + + [Fact] + public async Task Runs_each_query_read_only_with_statement_and_lock_timeouts() + { + var row = Assert.Single(await QueryAsync(""" + SELECT current_setting('transaction_read_only') AS read_only, + current_setting('statement_timeout') AS statement_timeout, + current_setting('lock_timeout') AS lock_timeout, + current_setting('application_name') AS application_name + """)); + + Assert.Equal("on", row["read_only"]); + Assert.Equal("5s", row["statement_timeout"]); + Assert.Equal("1s", row["lock_timeout"]); + Assert.Equal("pgcheckup", row["application_name"]); + } + + [Fact] + public async Task Ends_the_transaction_after_each_query() + { + await using var session = await ReadOnlySession.OpenAsync(postgres.Server.Checkup, Cancel); + + var first = Assert.Single(await session.QueryAsync("SELECT now() AS started", [], Cancel))["started"]; + var second = Assert.Single(await session.QueryAsync("SELECT now() AS started", [], Cancel))["started"]; + + Assert.NotEqual(first, second); + } + + [Fact] + public async Task Sets_nothing_for_the_whole_session() + { + await using var session = await ReadOnlySession.OpenAsync(postgres.Server.Checkup, Cancel); + await session.QueryAsync("SELECT 1 AS one", [], Cancel); + + // Behind a transaction pooler, anything set for the session reaches the app's next + // transaction. reset_val shows settings sent when connecting. + var row = Assert.Single(await session.QueryAsync(""" + SELECT current_setting('default_transaction_read_only') AS read_only, + (SELECT reset_val FROM pg_settings WHERE name = 'statement_timeout') AS statement_timeout, + (SELECT reset_val FROM pg_settings WHERE name = 'lock_timeout') AS lock_timeout + """, [], Cancel)); + + Assert.Equal("off", row["read_only"]); + Assert.Equal("0", row["statement_timeout"]); + Assert.Equal("0", row["lock_timeout"]); + } + + [Fact] + public async Task Rejects_a_write_even_when_the_role_may_write() + { + await postgres.Server.ExecuteAsSuperuserAsync(Cancel, "CREATE TABLE written (n int)", "GRANT INSERT ON written TO checkup"); + + var error = await Assert.ThrowsAsync(() => + QueryAsync("WITH w AS (INSERT INTO written VALUES (1) RETURNING n) SELECT n FROM w")); + + Assert.Equal(PostgresErrorCodes.ReadOnlySqlTransaction, error.SqlState); + } + + [Fact] + public async Task Rejects_a_second_statement() + { + var error = await Assert.ThrowsAsync(() => QueryAsync("SELECT 1; SELECT 2")); + + Assert.Equal(PostgresErrorCodes.SyntaxError, error.SqlState); + } + + [Fact] + public async Task Gives_up_on_a_lock_instead_of_queueing_behind_it() + { + await postgres.Server.ExecuteAsSuperuserAsync(Cancel, "CREATE TABLE migrating (n int)", "GRANT SELECT ON migrating TO checkup"); + await using var migration = await postgres.Server.OpenSuperuserAsync(Cancel); + await using var transaction = await migration.BeginTransactionAsync(Cancel); + await using (var lockTable = new NpgsqlCommand("LOCK TABLE migrating IN ACCESS EXCLUSIVE MODE", migration, transaction)) + { + await lockTable.ExecuteNonQueryAsync(Cancel); + } + + var error = await Assert.ThrowsAsync(() => QueryAsync("SELECT count(*) AS n FROM migrating")); + + Assert.Equal(PostgresErrorCodes.LockNotAvailable, error.SqlState); + } + + [Fact] + public async Task Binds_threshold_parameters_by_position() + { + var row = Assert.Single(await QueryAsync( + "SELECT $1 AS bytes, $2 AS age, $3 AS ratio", + 1_073_741_824L, + TimeSpan.FromHours(1), + 0.9m)); + + Assert.Equal(1_073_741_824L, row["bytes"]); + Assert.Equal(TimeSpan.FromHours(1), row["age"]); + Assert.Equal(0.9m, row["ratio"]); + } + + [Fact] + public async Task Returns_null_for_sql_null() + { + var row = Assert.Single(await QueryAsync("SELECT NULL::interval AS missing")); + + Assert.Null(row["missing"]); + } +} diff --git a/tests/Pgcheckup.Tests/Postgres/PostgresServer.cs b/tests/Pgcheckup.Tests/Postgres/PostgresServer.cs new file mode 100644 index 0000000..8cc6521 --- /dev/null +++ b/tests/Pgcheckup.Tests/Postgres/PostgresServer.cs @@ -0,0 +1,63 @@ +using Npgsql; +using Testcontainers.PostgreSql; + +namespace Pgcheckup.Tests.Postgres; + +// A throwaway Postgres with the placeholder database `app` and a `checkup` role that has only +// pg_monitor, the least privilege pgcheckup promises to need. +public sealed class PostgresServer : IAsyncDisposable +{ + private readonly PostgreSqlContainer container; + + private PostgresServer(PostgreSqlContainer container) => this.container = container; + + // CI runs the suite once per supported major, for example PGCHECKUP_TEST_POSTGRES=14. + public static string Version => + Environment.GetEnvironmentVariable("PGCHECKUP_TEST_POSTGRES") is { Length: > 0 } version ? version : "18"; + + public string SuperuserConnectionString => container.GetConnectionString(); + + public NpgsqlConnectionStringBuilder Checkup => new(container.GetConnectionString()) + { + Username = "checkup", + Password = "checkup", + }; + + public static async Task StartAsync(CancellationToken cancellationToken) + { + var container = new PostgreSqlBuilder($"postgres:{Version}-alpine").WithDatabase("app").Build(); + await container.StartAsync(cancellationToken); + var server = new PostgresServer(container); + await server.ExecuteAsSuperuserAsync(cancellationToken, "CREATE ROLE checkup LOGIN PASSWORD 'checkup' IN ROLE pg_monitor"); + return server; + } + + public async Task OpenSuperuserAsync(CancellationToken cancellationToken) + { + var connection = new NpgsqlConnection(SuperuserConnectionString); + await connection.OpenAsync(cancellationToken); + return connection; + } + + // One statement per string: SQL rewriting is off in tests too, as it is in pgcheckup. + public async Task ExecuteAsSuperuserAsync(CancellationToken cancellationToken, params string[] statements) + { + await using var connection = await OpenSuperuserAsync(cancellationToken); + foreach (var sql in statements) + { + await using var command = new NpgsqlCommand(sql, connection); + await command.ExecuteNonQueryAsync(cancellationToken); + } + } + + public ValueTask DisposeAsync() => container.DisposeAsync(); +} + +public sealed class PostgresServerFixture : IAsyncLifetime +{ + public PostgresServer Server { get; private set; } = null!; + + public async ValueTask InitializeAsync() => Server = await PostgresServer.StartAsync(TestContext.Current.CancellationToken); + + public ValueTask DisposeAsync() => Server.DisposeAsync(); +} From 4e4119faacf33687cc5350e3477893f08cfa7b12 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:00:02 -0500 Subject: [PATCH 07/26] feat(engine): turn check rows into findings --- src/Pgcheckup/Engine/CheckRunner.cs | 60 ++++++++++++ .../Engine/CheckRunnerTests.cs | 93 +++++++++++++++++++ 2 files changed, 153 insertions(+) create mode 100644 src/Pgcheckup/Engine/CheckRunner.cs create mode 100644 tests/Pgcheckup.Tests/Engine/CheckRunnerTests.cs diff --git a/src/Pgcheckup/Engine/CheckRunner.cs b/src/Pgcheckup/Engine/CheckRunner.cs new file mode 100644 index 0000000..50fda5c --- /dev/null +++ b/src/Pgcheckup/Engine/CheckRunner.cs @@ -0,0 +1,60 @@ +using Pgcheckup.Checks; + +namespace Pgcheckup.Engine; + +public sealed record Finding( + string CheckId, + string Subject, + Severity Severity, + string Message, + string Fix, + IReadOnlyDictionary Values); + +// A check whose query or template doesn't hold up its side of the contract. +public sealed class CheckException(string message, Exception? inner = null) : Exception(message, inner); + +public static class CheckRunner +{ + public static async Task> RunAsync(ReadOnlySession session, CheckDefinition check, CancellationToken cancellationToken) + { + var parameters = check.Thresholds.Select(ParameterValue).ToList(); + var rows = await session.QueryAsync(check.Sql, parameters, cancellationToken); + return rows.Select(row => ToFinding(check, row)).ToList(); + } + + private static object ParameterValue(Threshold threshold) => threshold.Kind switch + { + ThresholdKind.Bytes or ThresholdKind.Integer => (long)threshold.Value, + ThresholdKind.Duration => TimeSpan.FromMicroseconds((long)threshold.Value), + _ => threshold.Value, + }; + + private static Finding ToFinding(CheckDefinition check, IReadOnlyDictionary row) + { + if (!row.TryGetValue("subject", out var subject) || subject is not string subjectText) + { + throw new CheckException($"{check.Id} returned a row without a subject."); + } + + var severity = check.Severity; + if (row.TryGetValue("severity", out var value) && value != null) + { + severity = value switch + { + "critical" => Severity.Critical, + "warning" => Severity.Warning, + "info" => Severity.Info, + _ => throw new CheckException($"{check.Id} returned the severity {value}. Use critical, warning or info."), + }; + } + + try + { + return new Finding(check.Id, subjectText, severity, check.Message.Render(row), check.Fix.Render(row), row); + } + catch (TemplateException error) + { + throw new CheckException($"{check.Id}: {error.Message}", error); + } + } +} diff --git a/tests/Pgcheckup.Tests/Engine/CheckRunnerTests.cs b/tests/Pgcheckup.Tests/Engine/CheckRunnerTests.cs new file mode 100644 index 0000000..89140a3 --- /dev/null +++ b/tests/Pgcheckup.Tests/Engine/CheckRunnerTests.cs @@ -0,0 +1,93 @@ +using Pgcheckup.Checks; +using Pgcheckup.Engine; +using Pgcheckup.Tests.Postgres; + +namespace Pgcheckup.Tests.Engine; + +public class CheckRunnerTests(PostgresServerFixture postgres) : IClassFixture +{ + private static CancellationToken Cancel => TestContext.Current.CancellationToken; + + private static CheckDefinition Check(string sql, params Threshold[] thresholds) => new( + Id: "sample-check", + Title: "Sample check", + Category: "wal", + Severity: Severity.Warning, + MinVersion: 14, + Privileges: [], + SkipOn: [], + Thresholds: thresholds, + Sql: sql, + Message: new Template([new TextPart("Thing "), new ValuePart("subject", ValueFormat.Default), new TextPart(" holds "), new ValuePart("size", ValueFormat.Bytes), new TextPart(".")]), + Fix: new Template([new TextPart("Drop "), new ValuePart("subject", ValueFormat.Default), new TextPart(".")]), + Note: ""); + + private async Task> RunAsync(CheckDefinition check) + { + await using var session = await ReadOnlySession.OpenAsync(postgres.Server.Checkup, Cancel); + return await CheckRunner.RunAsync(session, check, Cancel); + } + + [Fact] + public async Task Reports_a_finding_per_row_with_its_message_and_fix() + { + var findings = await RunAsync(Check("SELECT 'a' AS subject, 1024::bigint AS size UNION ALL SELECT 'b', 2048")); + + Assert.Collection( + findings, + f => + { + Assert.Equal("sample-check", f.CheckId); + Assert.Equal("a", f.Subject); + Assert.Equal(Severity.Warning, f.Severity); + Assert.Equal("Thing a holds 1 kB.", f.Message); + Assert.Equal("Drop a.", f.Fix); + }, + f => Assert.Equal("Thing b holds 2 kB.", f.Message)); + } + + [Fact] + public async Task Reports_nothing_when_the_query_returns_no_rows() + { + Assert.Empty(await RunAsync(Check("SELECT 'a' AS subject, 1 AS size WHERE false"))); + } + + [Fact] + public async Task Lets_a_row_raise_its_severity() + { + var findings = await RunAsync(Check( + "SELECT 'a' AS subject, 1 AS size, 'critical' AS severity UNION ALL SELECT 'b', 1, NULL")); + + Assert.Equal([Severity.Critical, Severity.Warning], findings.Select(f => f.Severity)); + } + + [Fact] + public async Task Binds_thresholds_as_typed_parameters_in_order() + { + var check = Check( + "SELECT 'a' AS subject, 1 AS size WHERE $1 = 1073741824::bigint AND $2 = interval '1 hour' AND $3 = 5 AND $4 = 0.9", + new Threshold("min_size", ThresholdKind.Bytes, 1_073_741_824m, "1GB"), + new Threshold("min_age", ThresholdKind.Duration, 3_600_000_000m, "1h"), + new Threshold("min_count", ThresholdKind.Integer, 5m, "5"), + new Threshold("min_ratio", ThresholdKind.Number, 0.9m, "0.9")); + + Assert.Single(await RunAsync(check)); + } + + [Fact] + public async Task Rejects_a_row_without_a_subject() + { + var error = await Assert.ThrowsAsync(() => RunAsync(Check("SELECT NULL::text AS subject, 1 AS size"))); + + Assert.Contains("subject", error.Message); + } + + [Fact] + public async Task Rejects_an_unknown_severity() + { + var error = await Assert.ThrowsAsync(() => + RunAsync(Check("SELECT 'a' AS subject, 1 AS size, 'high' AS severity"))); + + Assert.Contains("high", error.Message); + } +} From 5c1898edd8cdcc175949b2a80cd45e5576bb42e1 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:00:02 -0500 Subject: [PATCH 08/26] feat(checks): add replication-slot-inactive --- checks/replication-slot-inactive/check.md | 35 +++++++++++++++++++ checks/replication-slot-inactive/check.sql | 18 ++++++++++ .../fixtures/fires.sql | 4 +++ .../fixtures/healthy.sql | 4 +++ 4 files changed, 61 insertions(+) create mode 100644 checks/replication-slot-inactive/check.md create mode 100644 checks/replication-slot-inactive/check.sql create mode 100644 checks/replication-slot-inactive/fixtures/fires.sql create mode 100644 checks/replication-slot-inactive/fixtures/healthy.sql diff --git a/checks/replication-slot-inactive/check.md b/checks/replication-slot-inactive/check.md new file mode 100644 index 0000000..2c013b6 --- /dev/null +++ b/checks/replication-slot-inactive/check.md @@ -0,0 +1,35 @@ +--- +id: replication-slot-inactive +title: Inactive replication slot +category: wal +severity: warning +min_version: 14 +privileges: [] +thresholds: + min_retained_wal: 1GB +message: Slot {subject} has been inactive[ for {inactive_for}] and is holding {retained_wal:bytes} of WAL. +fix: | + restart its consumer, or drop the slot: + SELECT pg_drop_replication_slot({slot_literal}); +--- + +## What breaks + +A replication slot makes Postgres keep every WAL segment that its consumer hasn't confirmed. When the consumer stops, the slot keeps WAL for as long as it exists. Typical consumers are a replica that was removed, a paused CDC connector such as Debezium, or a subscription dropped without its slot. `pg_wal` grows until the disk is full, and then Postgres stops accepting writes. + +A logical slot also holds back `catalog_xmin`, so vacuum can't clean up the system catalogs while the slot waits. + +## Fix + +If the consumer should still exist, restart it and let it catch up. If it shouldn't, drop the slot: + +```sql +SELECT pg_drop_replication_slot('slot_name'); +``` + +Then cap the WAL any slot can keep with `max_slot_wal_keep_size`. A slot that passes the cap is invalidated instead of filling the disk. On Postgres 18, `idle_replication_slot_timeout` also invalidates slots that stay inactive for too long. + +## Seen in + +- [The Insatiable Postgres Replication Slot](https://www.morling.dev/blog/insatiable-postgres-replication-slot/), Gunnar Morling: an inactive slot on an idle Amazon RDS database kept growing its WAL. +- [Replication slots](https://www.postgresql.org/docs/current/warm-standby.html#STREAMING-REPLICATION-SLOTS), PostgreSQL documentation: "replication slots can cause the server to retain so many WAL segments that they fill up the space allocated for pg_wal." diff --git a/checks/replication-slot-inactive/check.sql b/checks/replication-slot-inactive/check.sql new file mode 100644 index 0000000..a5e3170 --- /dev/null +++ b/checks/replication-slot-inactive/check.sql @@ -0,0 +1,18 @@ +SELECT s.slot_name AS subject, + quote_literal(s.slot_name) AS slot_literal, + -- inactive_since arrived in Postgres 17. Reading it through to_jsonb keeps one query + -- for every supported version, and gives NULL before 17. + now() - (to_jsonb(s) ->> 'inactive_since')::timestamptz AS inactive_for, + w.retained_wal +FROM pg_replication_slots AS s +CROSS JOIN LATERAL ( + -- On a standby, pg_current_wal_lsn() raises an error; the replay position is its equivalent. + SELECT pg_wal_lsn_diff( + CASE WHEN pg_is_in_recovery() THEN pg_last_wal_replay_lsn() ELSE pg_current_wal_lsn() END, + s.restart_lsn)::bigint AS retained_wal +) AS w +WHERE NOT s.active + -- A lost slot has already been invalidated and holds no WAL. + AND s.wal_status IS DISTINCT FROM 'lost' + AND w.retained_wal >= @min_retained_wal +ORDER BY w.retained_wal DESC diff --git a/checks/replication-slot-inactive/fixtures/fires.sql b/checks/replication-slot-inactive/fixtures/fires.sql new file mode 100644 index 0000000..0bb7edc --- /dev/null +++ b/checks/replication-slot-inactive/fixtures/fires.sql @@ -0,0 +1,4 @@ +-- threshold min_retained_wal = 0B +-- A physical slot that reserves WAL from the start and never gets a consumer. +SELECT pg_create_physical_replication_slot('fixture_slot', true); +CREATE TABLE fixture_wal AS SELECT g FROM generate_series(1, 1000) AS g; diff --git a/checks/replication-slot-inactive/fixtures/healthy.sql b/checks/replication-slot-inactive/fixtures/healthy.sql new file mode 100644 index 0000000..74b0433 --- /dev/null +++ b/checks/replication-slot-inactive/fixtures/healthy.sql @@ -0,0 +1,4 @@ +-- threshold min_retained_wal = 0B +-- A slot that has never reserved WAL holds none back, so even a zero threshold stays quiet. +SELECT pg_create_physical_replication_slot('fixture_slot'); +CREATE TABLE fixture_wal AS SELECT g FROM generate_series(1, 1000) AS g; From 38d21d73f06c44e9ce193356f852b30361b7fbf8 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:00:03 -0500 Subject: [PATCH 09/26] test(checks): run every check's fixtures on Testcontainers --- src/Pgcheckup/Pgcheckup.csproj | 3 +- tests/Pgcheckup.Tests/Checks/FixtureTests.cs | 59 ++++++++++++++ tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj | 3 + .../Pgcheckup.Tests/Postgres/FixtureScript.cs | 80 +++++++++++++++++++ 4 files changed, 143 insertions(+), 2 deletions(-) create mode 100644 tests/Pgcheckup.Tests/Checks/FixtureTests.cs create mode 100644 tests/Pgcheckup.Tests/Postgres/FixtureScript.cs diff --git a/src/Pgcheckup/Pgcheckup.csproj b/src/Pgcheckup/Pgcheckup.csproj index 3698197..da6ad63 100644 --- a/src/Pgcheckup/Pgcheckup.csproj +++ b/src/Pgcheckup/Pgcheckup.csproj @@ -17,7 +17,6 @@ - - + diff --git a/tests/Pgcheckup.Tests/Checks/FixtureTests.cs b/tests/Pgcheckup.Tests/Checks/FixtureTests.cs new file mode 100644 index 0000000..afe4217 --- /dev/null +++ b/tests/Pgcheckup.Tests/Checks/FixtureTests.cs @@ -0,0 +1,59 @@ +using Npgsql; +using Pgcheckup.Checks; +using Pgcheckup.Engine; +using Pgcheckup.Tests.Postgres; + +namespace Pgcheckup.Tests.Checks; + +// Every check runs against its own fixtures on a fresh Postgres, as the checkup role, through +// the same session and runner as a scan. `fires` must produce a finding and `healthy` must not. +public class FixtureTests +{ + private static CancellationToken Cancel => TestContext.Current.CancellationToken; + + public static TheoryData Fixtures() + { + var data = new TheoryData(); + foreach (var check in CheckCatalog.All) + { + data.Add(check.Id, "fires"); + data.Add(check.Id, "healthy"); + } + + return data; + } + + [Theory] + [MemberData(nameof(Fixtures))] + public async Task Fires_on_its_fires_fixture_and_stays_quiet_on_healthy(string checkId, string fixture) + { + var check = CheckCatalog.All.Single(c => c.Id == checkId); + if (int.Parse(PostgresServer.Version) < check.MinVersion) + { + Assert.Skip($"{checkId} needs Postgres {check.MinVersion} or later."); + } + + var script = FixtureScript.Load(checkId, fixture); + await using var server = await PostgresServer.StartAsync(Cancel); + + // Stays open until the check has run, so a fixture can hold a transaction or lock open. + await using var setup = await server.OpenSuperuserAsync(Cancel); + foreach (var statement in script.Statements) + { + await using var command = new NpgsqlCommand(statement, setup); + await command.ExecuteNonQueryAsync(Cancel); + } + + await using var session = await ReadOnlySession.OpenAsync(server.Checkup, Cancel); + var findings = await CheckRunner.RunAsync(session, script.Apply(check), Cancel); + + if (fixture == "fires") + { + Assert.NotEmpty(findings); + } + else + { + Assert.Empty(findings); + } + } +} diff --git a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj index 93772b8..dbff69f 100644 --- a/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj +++ b/tests/Pgcheckup.Tests/Pgcheckup.Tests.csproj @@ -15,4 +15,7 @@ + + + diff --git a/tests/Pgcheckup.Tests/Postgres/FixtureScript.cs b/tests/Pgcheckup.Tests/Postgres/FixtureScript.cs new file mode 100644 index 0000000..46491f8 --- /dev/null +++ b/tests/Pgcheckup.Tests/Postgres/FixtureScript.cs @@ -0,0 +1,80 @@ +using System.Text.RegularExpressions; +using Pgcheckup.Checks; +using Pgcheckup.Checks.Generator; +using RuntimeThresholdKind = Pgcheckup.Checks.ThresholdKind; + +namespace Pgcheckup.Tests.Postgres; + +// A check's fixture: setup statements, plus `-- threshold name = value` lines that lower a +// threshold when the real condition can't be reproduced at full scale. +internal sealed partial class FixtureScript +{ + private FixtureScript(IReadOnlyList statements, IReadOnlyDictionary thresholds) + { + Statements = statements; + Thresholds = thresholds; + } + + public IReadOnlyList Statements { get; } + + public IReadOnlyDictionary Thresholds { get; } + + public static FixtureScript Load(string checkId, string fixture) => + Parse(File.ReadAllText(Path.Combine(AppContext.BaseDirectory, "checks", checkId, "fixtures", fixture + ".sql"))); + + public static FixtureScript Parse(string text) + { + var thresholds = ThresholdLine().Matches(text).ToDictionary(m => m.Groups[1].Value, m => m.Groups[2].Value.Trim()); + + var errors = new List(); + var tokens = SqlTokenizer.Tokenize(text, errors); + if (errors.Count > 0) + { + throw new InvalidOperationException($"The fixture doesn't parse: {string.Join("; ", errors)}"); + } + + var statements = new List(); + var start = 0; + foreach (var end in tokens.Where(t => t.Kind == TokenKind.Semicolon).Select(t => t.Start).Append(text.Length)) + { + if (tokens.Any(t => t.Start >= start && t.Start < end && t.Kind != TokenKind.Semicolon)) + { + statements.Add(text[start..end].Trim()); + } + + start = end + 1; + } + + return new FixtureScript(statements, thresholds); + } + + public CheckDefinition Apply(CheckDefinition check) + { + var unknown = Thresholds.Keys.Except(check.Thresholds.Select(t => t.Name)).ToList(); + if (unknown.Count > 0) + { + throw new InvalidOperationException($"The fixture lowers {string.Join(", ", unknown)}, which {check.Id} doesn't have."); + } + + return check with + { + Thresholds = check.Thresholds.Select(t => + { + if (!Thresholds.TryGetValue(t.Name, out var text)) + { + return t; + } + + if (!ThresholdValue.TryParse(text, out var value, out var error) || value.Kind.ToString() != t.Kind.ToString()) + { + throw new InvalidOperationException($"The fixture sets {t.Name} to {text}, which isn't a {t.Kind} value. {error}"); + } + + return t with { Value = value.Value, Text = text }; + }).ToList(), + }; + } + + [GeneratedRegex(@"^--\s*threshold\s+([a-z][a-z0-9_]*)\s*=\s*(.+)$", RegexOptions.Multiline)] + private static partial Regex ThresholdLine(); +} From 46385bcd96bfe7e6e0bc5a3983e7df63cf5cb661 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:14:47 -0500 Subject: [PATCH 10/26] feat(cli): read connections from URLs, key-value strings and PG variables --- src/Pgcheckup/Cli/ConnectionInput.cs | 279 ++++++++++++++++++ .../Cli/ConnectionInputTests.cs | 137 +++++++++ 2 files changed, 416 insertions(+) create mode 100644 src/Pgcheckup/Cli/ConnectionInput.cs create mode 100644 tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs diff --git a/src/Pgcheckup/Cli/ConnectionInput.cs b/src/Pgcheckup/Cli/ConnectionInput.cs new file mode 100644 index 0000000..739bb14 --- /dev/null +++ b/src/Pgcheckup/Cli/ConnectionInput.cs @@ -0,0 +1,279 @@ +using System.Globalization; +using System.Text; +using Npgsql; + +namespace Pgcheckup.Cli; + +// Errors name the part that is wrong and never repeat the input, which may hold a password. +public sealed class ConnectionInputException(string message) : Exception(message); + +// Reads connections the way psql does: a postgres:// URL, a libpq key-value string, or nothing, +// with PGHOST, PGPORT, PGDATABASE and PGSSLMODE filling in what the input leaves out. Npgsql +// reads PGUSER, PGPASSWORD, PGPASSFILE and ~/.pgpass itself. +public static class ConnectionInput +{ + private static readonly (string Variable, string Key)[] Environment = + [ + ("PGHOST", "host"), + ("PGPORT", "port"), + ("PGDATABASE", "dbname"), + ("PGSSLMODE", "sslmode"), + ]; + + private static readonly Dictionary SslModes = new(StringComparer.Ordinal) + { + ["disable"] = SslMode.Disable, + ["allow"] = SslMode.Allow, + ["prefer"] = SslMode.Prefer, + ["require"] = SslMode.Require, + ["verify-ca"] = SslMode.VerifyCA, + ["verify-full"] = SslMode.VerifyFull, + }; + + private static readonly Dictionary> Keywords = new(StringComparer.Ordinal) + { + ["host"] = (b, v) => b.Host = v, + ["port"] = (b, v) => b.Port = Number("port", v), + ["dbname"] = (b, v) => b.Database = v, + ["user"] = (b, v) => b.Username = v, + ["password"] = (b, v) => b.Password = v, + ["passfile"] = (b, v) => b.Passfile = v, + ["sslmode"] = (b, v) => b.SslMode = SslModes.TryGetValue(v, out var mode) + ? mode + : throw new ConnectionInputException($"sslmode {v} isn't one of {string.Join(", ", SslModes.Keys)}."), + ["sslrootcert"] = (b, v) => b.RootCertificate = v, + ["sslcert"] = (b, v) => b.SslCertificate = v, + ["sslkey"] = (b, v) => b.SslKey = v, + ["sslpassword"] = (b, v) => b.SslPassword = v, + ["connect_timeout"] = (b, v) => b.Timeout = Number("connect_timeout", v), + ["target_session_attrs"] = (b, v) => b.TargetSessionAttributes = v, + ["options"] = (b, v) => b.Options = v, + + // pgcheckup always connects as application_name=pgcheckup. + ["application_name"] = (_, _) => { }, + ["fallback_application_name"] = (_, _) => { }, + }; + + public static NpgsqlConnectionStringBuilder Parse(string? input, IReadOnlyDictionary environment) + { + var values = new Dictionary(StringComparer.Ordinal); + if (!string.IsNullOrWhiteSpace(input)) + { + input = input.Trim(); + if (input.StartsWith("postgres://", StringComparison.Ordinal) || input.StartsWith("postgresql://", StringComparison.Ordinal)) + { + ReadUrl(input, values); + } + else if (input.Contains("://", StringComparison.Ordinal)) + { + throw new ConnectionInputException("A connection URL must start with postgres:// or postgresql://."); + } + else + { + ReadKeyValues(input, values); + } + } + + foreach (var (variable, key) in Environment) + { + if (!values.ContainsKey(key) && environment.TryGetValue(variable, out var value) && !string.IsNullOrEmpty(value)) + { + values[key] = value; + } + } + + // libpq's fallback when there's no Unix socket. Npgsql has no default at all. + values.TryAdd("host", "localhost"); + + var unknown = values.Keys.FirstOrDefault(k => !Keywords.ContainsKey(k)); + if (unknown != null) + { + throw new ConnectionInputException($"pgcheckup doesn't know the connection parameter {unknown}."); + } + + var settings = new NpgsqlConnectionStringBuilder(); + foreach (var (key, value) in values) + { + Keywords[key](settings, value); + } + + return settings; + } + + private static void ReadUrl(string url, Dictionary values) + { + var rest = url[(url.IndexOf("://", StringComparison.Ordinal) + 3)..]; + var query = ""; + var questionMark = rest.IndexOf('?'); + if (questionMark >= 0) + { + query = rest[(questionMark + 1)..]; + rest = rest[..questionMark]; + } + + var slash = rest.IndexOf('/'); + var authority = slash >= 0 ? rest[..slash] : rest; + var database = slash >= 0 ? rest[(slash + 1)..] : ""; + if (database.Length > 0) + { + values["dbname"] = Uri.UnescapeDataString(database); + } + + var at = authority.LastIndexOf('@'); + if (at >= 0) + { + var userInfo = authority[..at]; + authority = authority[(at + 1)..]; + var colon = userInfo.IndexOf(':'); + values["user"] = Uri.UnescapeDataString(colon >= 0 ? userInfo[..colon] : userInfo); + if (colon >= 0) + { + values["password"] = Uri.UnescapeDataString(userInfo[(colon + 1)..]); + } + } + + if (authority.Length > 0) + { + var hosts = authority.Split(',').Select(SplitHostPort).ToList(); + if (hosts.Count == 1) + { + values["host"] = hosts[0].Host; + if (hosts[0].Port != null) + { + values["port"] = hosts[0].Port!; + } + } + else + { + values["host"] = string.Join(",", hosts.Select(h => h.Port == null ? h.Host : $"{h.Host}:{h.Port}")); + } + } + + foreach (var pair in query.Split('&', StringSplitOptions.RemoveEmptyEntries)) + { + var equals = pair.IndexOf('='); + var key = Uri.UnescapeDataString(equals >= 0 ? pair[..equals] : pair); + values[key] = Uri.UnescapeDataString(equals >= 0 ? pair[(equals + 1)..] : ""); + } + } + + private static (string Host, string? Port) SplitHostPort(string hostPort) + { + string host; + string? port = null; + if (hostPort.StartsWith('[')) + { + var close = hostPort.IndexOf(']'); + if (close < 0) + { + throw new ConnectionInputException("An IPv6 host in the URL is missing its closing ]."); + } + + host = hostPort[1..close]; + if (hostPort.Length > close + 1 && hostPort[close + 1] == ':') + { + port = hostPort[(close + 2)..]; + } + } + else + { + var colon = hostPort.LastIndexOf(':'); + host = colon >= 0 ? hostPort[..colon] : hostPort; + port = colon >= 0 ? hostPort[(colon + 1)..] : null; + } + + if (port != null) + { + Number("port", port); + } + + return (Uri.UnescapeDataString(host), port); + } + + // libpq's format: key=value pairs separated by spaces, with values in single quotes when + // they hold spaces, and backslash escapes inside. + private static void ReadKeyValues(string input, Dictionary values) + { + var i = 0; + while (i < input.Length) + { + if (char.IsWhiteSpace(input[i])) + { + i++; + continue; + } + + var keyStart = i; + while (i < input.Length && input[i] != '=' && !char.IsWhiteSpace(input[i])) + { + i++; + } + + var key = input[keyStart..i]; + while (i < input.Length && char.IsWhiteSpace(input[i])) + { + i++; + } + + if (i >= input.Length || input[i] != '=') + { + throw new ConnectionInputException($"The connection parameter {key} has no = and value."); + } + + i++; + while (i < input.Length && char.IsWhiteSpace(input[i])) + { + i++; + } + + var value = new StringBuilder(); + if (i < input.Length && input[i] == '\'') + { + i++; + var closed = false; + while (i < input.Length) + { + if (input[i] == '\\' && i + 1 < input.Length) + { + value.Append(input[i + 1]); + i += 2; + } + else if (input[i] == '\'') + { + closed = true; + i++; + break; + } + else + { + value.Append(input[i++]); + } + } + + if (!closed) + { + throw new ConnectionInputException($"The value of {key} opens a quote that never closes."); + } + } + else + { + while (i < input.Length && !char.IsWhiteSpace(input[i])) + { + if (input[i] == '\\' && i + 1 < input.Length) + { + i++; + } + + value.Append(input[i++]); + } + } + + values[key] = value.ToString(); + } + } + + private static int Number(string key, string value) => + int.TryParse(value, NumberStyles.None, CultureInfo.InvariantCulture, out var number) + ? number + : throw new ConnectionInputException($"The {key} {value} isn't a number."); +} diff --git a/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs b/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs new file mode 100644 index 0000000..ac44c93 --- /dev/null +++ b/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs @@ -0,0 +1,137 @@ +using Npgsql; +using Pgcheckup.Cli; + +namespace Pgcheckup.Tests.Cli; + +public class ConnectionInputTests +{ + private static readonly Dictionary NoEnvironment = []; + + private static NpgsqlConnectionStringBuilder Parse(string? input, Dictionary? environment = null) => + ConnectionInput.Parse(input, environment ?? NoEnvironment); + + [Fact] + public void Reads_a_postgres_url() + { + var settings = Parse("postgres://checkup:s%40cret@db.example.com:5433/app?sslmode=verify-full"); + + Assert.Equal("db.example.com", settings.Host); + Assert.Equal(5433, settings.Port); + Assert.Equal("app", settings.Database); + Assert.Equal("checkup", settings.Username); + Assert.Equal("s@cret", settings.Password); + Assert.Equal(SslMode.VerifyFull, settings.SslMode); + } + + [Fact] + public void Reads_a_postgresql_url_without_user_or_port() + { + var settings = Parse("postgresql://db.example.com/app"); + + Assert.Equal("db.example.com", settings.Host); + Assert.Equal(5432, settings.Port); + Assert.Equal("app", settings.Database); + Assert.Null(settings.Username); + } + + [Fact] + public void Reads_ipv6_and_several_hosts() + { + Assert.Equal("::1", Parse("postgres://[::1]:5432/app").Host); + Assert.Equal("h1.example.com:5432,h2.example.com:5433", Parse("postgres://h1.example.com:5432,h2.example.com:5433/app").Host); + } + + [Fact] + public void Reads_a_socket_directory_from_the_host_parameter() + { + var settings = Parse("postgres:///app?host=%2Fvar%2Frun%2Fpostgresql"); + + Assert.Equal("/var/run/postgresql", settings.Host); + Assert.Equal("app", settings.Database); + } + + [Fact] + public void Maps_libpq_parameters_to_npgsql() + { + var settings = Parse("postgres://db.example.com/app?connect_timeout=10&target_session_attrs=read-write&sslrootcert=ca.pem&sslcert=client.pem&sslkey=client.key&passfile=pass.conf"); + + Assert.Equal(10, settings.Timeout); + Assert.Equal("read-write", settings.TargetSessionAttributes); + Assert.Equal("ca.pem", settings.RootCertificate); + Assert.Equal("client.pem", settings.SslCertificate); + Assert.Equal("client.key", settings.SslKey); + Assert.Equal("pass.conf", settings.Passfile); + } + + [Fact] + public void Ignores_the_application_name_because_pgcheckup_sets_its_own() + { + Assert.Null(Parse("postgres://db.example.com/app?application_name=myapp").ApplicationName); + } + + [Fact] + public void Reads_a_libpq_key_value_string() + { + var settings = Parse("host=db.example.com port=5433 dbname=app user=checkup sslmode=require password='a b\\'c'"); + + Assert.Equal("db.example.com", settings.Host); + Assert.Equal(5433, settings.Port); + Assert.Equal("app", settings.Database); + Assert.Equal("checkup", settings.Username); + Assert.Equal(SslMode.Require, settings.SslMode); + Assert.Equal("a b'c", settings.Password); + } + + [Fact] + public void Fills_what_the_input_leaves_out_from_pg_environment_variables() + { + var environment = new Dictionary + { + ["PGHOST"] = "db.example.com", + ["PGPORT"] = "6543", + ["PGDATABASE"] = "app", + ["PGSSLMODE"] = "require", + }; + + var fromEnvironment = Parse(null, environment); + Assert.Equal("db.example.com", fromEnvironment.Host); + Assert.Equal(6543, fromEnvironment.Port); + Assert.Equal("app", fromEnvironment.Database); + Assert.Equal(SslMode.Require, fromEnvironment.SslMode); + + var mixed = Parse("postgres://other.example.com/app", environment); + Assert.Equal("other.example.com", mixed.Host); + Assert.Equal(6543, mixed.Port); + } + + [Fact] + public void Connects_to_localhost_when_nothing_names_a_host() + { + Assert.Equal("localhost", Parse(null).Host); + Assert.Equal("localhost", Parse("dbname=app").Host); + } + + [Theory] + [InlineData("postgres://db.example.com/app?colour=blue", "colour")] + [InlineData("host=db.example.com colour=blue", "colour")] + [InlineData("postgres://db.example.com/app?sslmode=sometimes", "sometimes")] + [InlineData("postgres://db.example.com:abc/app", "abc")] + [InlineData("mysql://db.example.com/app", "postgres://")] + [InlineData("host=db.example.com password='unterminated", "quote")] + public void Explains_what_is_wrong_with_the_input(string input, string named) + { + var error = Assert.Throws(() => Parse(input)); + + Assert.Contains(named, error.Message); + } + + [Theory] + [InlineData("postgres://checkup:hunter2@db.example.com:abc/app")] + [InlineData("host=db.example.com password=hunter2 colour=blue")] + public void Never_repeats_the_password_in_an_error(string input) + { + var error = Assert.Throws(() => Parse(input)); + + Assert.DoesNotContain("hunter2", error.Message); + } +} From 51e196ada3cde39a202ec8e9e120f551833dc991 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:14:48 -0500 Subject: [PATCH 11/26] feat(cli): add scan and list with terminal output and exit codes --- src/Pgcheckup/Cli/PgcheckupCli.cs | 131 ++++++++++++++++++ src/Pgcheckup/Cli/TerminalReport.cs | 90 ++++++++++++ src/Pgcheckup/Engine/Scanner.cs | 54 ++++++++ src/Pgcheckup/Program.cs | 16 ++- tests/Pgcheckup.Tests/Cli/CommandLineTests.cs | 84 +++++++++++ .../Cli/TerminalReportTests.cs | 120 ++++++++++++++++ .../Postgres/InactiveSlotFixture.cs | 43 ++++++ 7 files changed, 535 insertions(+), 3 deletions(-) create mode 100644 src/Pgcheckup/Cli/PgcheckupCli.cs create mode 100644 src/Pgcheckup/Cli/TerminalReport.cs create mode 100644 src/Pgcheckup/Engine/Scanner.cs create mode 100644 tests/Pgcheckup.Tests/Cli/CommandLineTests.cs create mode 100644 tests/Pgcheckup.Tests/Cli/TerminalReportTests.cs create mode 100644 tests/Pgcheckup.Tests/Postgres/InactiveSlotFixture.cs diff --git a/src/Pgcheckup/Cli/PgcheckupCli.cs b/src/Pgcheckup/Cli/PgcheckupCli.cs new file mode 100644 index 0000000..b0ac1ad --- /dev/null +++ b/src/Pgcheckup/Cli/PgcheckupCli.cs @@ -0,0 +1,131 @@ +using System.CommandLine; +using Npgsql; +using Pgcheckup.Checks; +using Pgcheckup.Engine; + +namespace Pgcheckup.Cli; + +public static class PgcheckupCli +{ + public const int Passed = 0; + public const int FindingsReachedFailOn = 1; + public const int CouldNotRun = 2; + + public static async Task RunAsync( + string[] args, + TextWriter output, + TextWriter error, + IReadOnlyDictionary environment, + bool outputRedirected, + CancellationToken cancellationToken) + { + var root = new RootCommand("Checks a PostgreSQL database for the problems that cause outages. Read-only, and safe to run on production."); + + var list = new Command("list", "List every check."); + list.SetAction(_ => List(output)); + root.Subcommands.Add(list); + + var connection = new Argument("connection") + { + Description = "A postgres:// URL or a libpq key-value string. Without one, the PG* environment variables are used. Keep the password in PGPASSWORD or ~/.pgpass, not here.", + Arity = ArgumentArity.ZeroOrOne, + }; + var failOn = new Option("--fail-on") + { + Description = "Exit with 1 when a finding reaches this severity: critical, warning or info.", + DefaultValueFactory = _ => "critical", + }; + failOn.AcceptOnlyFromAmong("critical", "warning", "info"); + + var scan = new Command("scan", "Scan a database and report what could take it down."); + scan.Arguments.Add(connection); + scan.Options.Add(failOn); + scan.SetAction((result, token) => ScanAsync( + result.GetValue(connection), result.GetValue(failOn)!, output, error, environment, outputRedirected, token)); + root.Subcommands.Add(scan); + + var parsed = root.Parse(args); + + // Exit code 1 means findings, so argument errors get 2 like any other scan that couldn't run. + if (parsed.Errors.Count > 0) + { + foreach (var parseError in parsed.Errors) + { + error.WriteLine($"pgcheckup: {parseError.Message}"); + } + + error.WriteLine("Run pgcheckup --help for usage."); + return CouldNotRun; + } + + return await parsed.InvokeAsync(new InvocationConfiguration { Output = output, Error = error }, cancellationToken); + } + + private static int List(TextWriter output) + { + var width = CheckCatalog.All.Max(c => c.Id.Length) + 2; + foreach (var check in CheckCatalog.All) + { + output.WriteLine($"{check.Id.PadRight(width)}{check.Severity.ToString().ToLowerInvariant(),-10}{check.Title}"); + } + + return Passed; + } + + private static async Task ScanAsync( + string? input, + string failOn, + TextWriter output, + TextWriter error, + IReadOnlyDictionary environment, + bool outputRedirected, + CancellationToken cancellationToken) + { + NpgsqlConnectionStringBuilder settings; + try + { + settings = ConnectionInput.Parse(input, environment); + } + catch (ConnectionInputException problem) + { + error.WriteLine($"pgcheckup: {problem.Message}"); + return CouldNotRun; + } + + var host = settings.Host ?? "localhost"; + ReadOnlySession session; + try + { + session = await ReadOnlySession.OpenAsync(settings, cancellationToken); + } + catch (Exception problem) when (problem is NpgsqlException or TimeoutException or ArgumentException or InvalidOperationException) + { + error.WriteLine($"pgcheckup: couldn't connect to {host}: {problem.Message}"); + return CouldNotRun; + } + + await using (session) + { + ScanReport report; + try + { + report = await Scanner.ScanAsync(session, host, CheckCatalog.All, cancellationToken); + } + catch (Exception problem) when (problem is CheckFailedException or NpgsqlException) + { + error.WriteLine($"pgcheckup: {problem.Message}"); + return CouldNotRun; + } + + TerminalReport.Write(output, report, TerminalReport.UseColor(outputRedirected, environment)); + + var threshold = failOn switch + { + "info" => Severity.Info, + "warning" => Severity.Warning, + _ => Severity.Critical, + }; + return report.Results.Any(r => r.Worst >= threshold) ? FindingsReachedFailOn : Passed; + } + } +} diff --git a/src/Pgcheckup/Cli/TerminalReport.cs b/src/Pgcheckup/Cli/TerminalReport.cs new file mode 100644 index 0000000..801b4bb --- /dev/null +++ b/src/Pgcheckup/Cli/TerminalReport.cs @@ -0,0 +1,90 @@ +using Pgcheckup.Checks; +using Pgcheckup.Engine; + +namespace Pgcheckup.Cli; + +public static class TerminalReport +{ + private const int Indent = 10; + private const string FixLabel = "Fix: "; + + public static bool UseColor(bool outputRedirected, IReadOnlyDictionary environment) => + !outputRedirected + && !(environment.TryGetValue("NO_COLOR", out var noColor) && !string.IsNullOrEmpty(noColor)) + && !(environment.TryGetValue("TERM", out var term) && term == "dumb"); + + public static void Write(TextWriter output, ScanReport report, bool color) + { + var server = report.Server; + output.WriteLine($"pgcheckup · {server.Database} on {server.Host} · PostgreSQL {server.Version}"); + output.WriteLine(); + + var findings = report.Results + .SelectMany(r => r.Findings) + .Select((finding, order) => (finding, order)) + .OrderByDescending(f => f.finding.Severity) + .ThenBy(f => f.finding.CheckId, StringComparer.Ordinal) + .ThenBy(f => f.order) + .Select(f => f.finding); + + foreach (var finding in findings) + { + var label = finding.Severity.ToString().ToUpperInvariant(); + output.WriteLine($"{Paint(label, finding.Severity, color)}{new string(' ', Indent - label.Length)}{finding.CheckId}"); + WriteIndented(output, finding.Message, new string(' ', Indent), new string(' ', Indent)); + WriteIndented(output, finding.Fix, new string(' ', Indent) + FixLabel, new string(' ', Indent + FixLabel.Length)); + output.WriteLine(); + } + + output.WriteLine(Summary(report)); + } + + private static void WriteIndented(TextWriter output, string text, string first, string rest) + { + var lines = text.Split('\n'); + for (var i = 0; i < lines.Length; i++) + { + output.WriteLine((i == 0 ? first : rest) + lines[i]); + } + } + + private static string Summary(ScanReport report) + { + var parts = new List { $"{report.Results.Count(r => r.Worst == null)} passed" }; + var critical = report.Results.Count(r => r.Worst == Severity.Critical); + var warning = report.Results.Count(r => r.Worst == Severity.Warning); + var info = report.Results.Count(r => r.Worst == Severity.Info); + if (critical > 0) + { + parts.Add($"{critical} critical"); + } + + if (warning > 0) + { + parts.Add(warning == 1 ? "1 warning" : $"{warning} warnings"); + } + + if (info > 0) + { + parts.Add($"{info} info"); + } + + return string.Join(" · ", parts); + } + + private static string Paint(string label, Severity severity, bool color) + { + if (!color) + { + return label; + } + + var code = severity switch + { + Severity.Critical => "1;31", + Severity.Warning => "1;33", + _ => "1;34", + }; + return $"\u001b[{code}m{label}\u001b[0m"; + } +} diff --git a/src/Pgcheckup/Engine/Scanner.cs b/src/Pgcheckup/Engine/Scanner.cs new file mode 100644 index 0000000..3331506 --- /dev/null +++ b/src/Pgcheckup/Engine/Scanner.cs @@ -0,0 +1,54 @@ +using System.Globalization; +using Pgcheckup.Checks; + +namespace Pgcheckup.Engine; + +public sealed record ServerInfo(string Database, string Host, string Version); + +public sealed record CheckResult(CheckDefinition Check, IReadOnlyList Findings) +{ + public Severity? Worst => Findings.Count == 0 ? null : Findings.Max(f => f.Severity); +} + +public sealed record ScanReport(ServerInfo Server, IReadOnlyList Results); + +// A check that couldn't run. In M0 this ends the scan; M1 reports it as errored and goes on. +public sealed class CheckFailedException(string checkId, Exception inner) + : Exception($"{checkId} couldn't run: {inner.Message}", inner) +{ + public string CheckId { get; } = checkId; +} + +public static class Scanner +{ + public static async Task ScanAsync( + ReadOnlySession session, string host, IReadOnlyList checks, CancellationToken cancellationToken) + { + var row = (await session.QueryAsync( + "SELECT current_database() AS database, current_setting('server_version_num')::int AS version", + [], + cancellationToken)).Single(); + + // server_version_num is major * 10000 + minor from Postgres 10 on. + var version = (int)row["version"]!; + var server = new ServerInfo( + (string)row["database"]!, + host, + string.Create(CultureInfo.InvariantCulture, $"{version / 10000}.{version % 10000}")); + + var results = new List(); + foreach (var check in checks) + { + try + { + results.Add(new CheckResult(check, await CheckRunner.RunAsync(session, check, cancellationToken))); + } + catch (Exception error) when (error is CheckException or Npgsql.NpgsqlException) + { + throw new CheckFailedException(check.Id, error); + } + } + + return new ScanReport(server, results); + } +} diff --git a/src/Pgcheckup/Program.cs b/src/Pgcheckup/Program.cs index e383c9e..629d969 100644 --- a/src/Pgcheckup/Program.cs +++ b/src/Pgcheckup/Program.cs @@ -1,4 +1,14 @@ -using System.CommandLine; +using System.Collections; +using System.Text; +using Pgcheckup.Cli; -var root = new RootCommand("Checks a PostgreSQL database for the problems that cause outages."); -return root.Parse(args).Invoke(); +// The report's separators and any non-ASCII object names need UTF-8, whatever the console's code page. +Console.OutputEncoding = new UTF8Encoding(encoderShouldEmitUTF8Identifier: false); + +var environment = new Dictionary(OperatingSystem.IsWindows() ? StringComparer.OrdinalIgnoreCase : StringComparer.Ordinal); +foreach (DictionaryEntry variable in Environment.GetEnvironmentVariables()) +{ + environment[(string)variable.Key] = (string?)variable.Value; +} + +return await PgcheckupCli.RunAsync(args, Console.Out, Console.Error, environment, Console.IsOutputRedirected, CancellationToken.None); diff --git a/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs b/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs new file mode 100644 index 0000000..ab8c096 --- /dev/null +++ b/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs @@ -0,0 +1,84 @@ +using Pgcheckup.Cli; +using Pgcheckup.Tests.Postgres; + +namespace Pgcheckup.Tests.Cli; + +public class CommandLineTests +{ + private static readonly Dictionary NoEnvironment = []; + + internal static async Task<(int ExitCode, string Output, string Error)> RunAsync(params string[] args) + { + var output = new StringWriter { NewLine = "\n" }; + var error = new StringWriter { NewLine = "\n" }; + var exitCode = await PgcheckupCli.RunAsync(args, output, error, NoEnvironment, outputRedirected: true, TestContext.Current.CancellationToken); + return (exitCode, output.ToString(), error.ToString()); + } + + [Fact] + public async Task Lists_every_check() + { + var (exitCode, output, _) = await RunAsync("list"); + + Assert.Equal(0, exitCode); + Assert.Contains("replication-slot-inactive", output); + Assert.Contains("Inactive replication slot", output); + } + + [Theory] + [InlineData("scan", "--nope")] + [InlineData("scan", "--fail-on", "sometimes")] + [InlineData("frobnicate")] + public async Task Exits_2_when_the_arguments_are_wrong(params string[] args) + { + var (exitCode, _, error) = await RunAsync(args); + + Assert.Equal(2, exitCode); + Assert.NotEmpty(error); + } + + [Fact] + public async Task Exits_2_when_the_connection_input_is_wrong() + { + var (exitCode, _, error) = await RunAsync("scan", "mysql://db.example.com/app"); + + Assert.Equal(2, exitCode); + Assert.Contains("postgres://", error); + } + + [Fact] + public async Task Exits_2_when_it_cannot_connect() + { + var (exitCode, _, error) = await RunAsync("scan", "postgres://checkup:hunter2@127.0.0.1:1/app?connect_timeout=2"); + + Assert.Equal(2, exitCode); + Assert.Contains("couldn't connect to 127.0.0.1", error); + Assert.DoesNotContain("hunter2", error); + } +} + +public class ScanCommandTests(InactiveSlotFixture postgres) : IClassFixture +{ + [Fact] + public async Task Reports_a_warning_and_exits_0_below_the_default_fail_on() + { + var server = await postgres.ServerAsync(); + var (exitCode, output, error) = await CommandLineTests.RunAsync("scan", await postgres.CheckupUrlAsync()); + + Assert.Equal("", error); + Assert.Equal(0, exitCode); + Assert.StartsWith($"pgcheckup · app on {server.Checkup.Host} · PostgreSQL {PostgresServer.Version}.", output); + Assert.Contains("WARNING replication-slot-inactive", output); + Assert.Contains("Slot debezium has been inactive", output); + Assert.Contains("SELECT pg_drop_replication_slot('debezium');", output); + Assert.EndsWith("0 passed · 1 warning\n", output); + } + + [Fact] + public async Task Exits_1_when_a_finding_reaches_fail_on() + { + var (exitCode, _, _) = await CommandLineTests.RunAsync("scan", await postgres.CheckupUrlAsync(), "--fail-on", "warning"); + + Assert.Equal(1, exitCode); + } +} diff --git a/tests/Pgcheckup.Tests/Cli/TerminalReportTests.cs b/tests/Pgcheckup.Tests/Cli/TerminalReportTests.cs new file mode 100644 index 0000000..590ce30 --- /dev/null +++ b/tests/Pgcheckup.Tests/Cli/TerminalReportTests.cs @@ -0,0 +1,120 @@ +using Pgcheckup.Checks; +using Pgcheckup.Cli; +using Pgcheckup.Engine; + +namespace Pgcheckup.Tests.Cli; + +public class TerminalReportTests +{ + private static readonly ServerInfo Server = new("app", "db.example.com", "17.6"); + + private static CheckDefinition Check(string id, Severity severity = Severity.Warning) => new( + id, id, "wal", severity, 14, [], [], [], "SELECT 1", new Template([]), new Template([]), ""); + + private static Finding Finding(string checkId, Severity severity, string message, string fix) => + new(checkId, "subject", severity, message, fix, new Dictionary()); + + private static string Render(ScanReport report, bool color = false) + { + var output = new StringWriter { NewLine = "\n" }; + TerminalReport.Write(output, report, color); + return output.ToString(); + } + + [Fact] + public void Writes_each_finding_with_its_fix_under_a_header_and_above_a_summary() + { + var slot = Check("replication-slot-inactive"); + var report = new ScanReport(Server, + [ + new CheckResult(Check("connection-saturation"), []), + new CheckResult(slot, + [ + Finding(slot.Id, Severity.Warning, + "Slot debezium has been inactive for 3 days and is holding 48 GB of WAL.", + "restart its consumer, or drop the slot:\nSELECT pg_drop_replication_slot('debezium');"), + ]), + ]); + + Assert.Equal( + """ + pgcheckup · app on db.example.com · PostgreSQL 17.6 + + WARNING replication-slot-inactive + Slot debezium has been inactive for 3 days and is holding 48 GB of WAL. + Fix: restart its consumer, or drop the slot: + SELECT pg_drop_replication_slot('debezium'); + + 1 passed · 1 warning + + """.ReplaceLineEndings("\n"), + Render(report)); + } + + [Fact] + public void Lists_critical_findings_first_and_counts_each_check_once_at_its_worst() + { + var slot = Check("replication-slot-inactive"); + var xid = Check("xid-wraparound", Severity.Critical); + var report = new ScanReport(Server, + [ + new CheckResult(slot, + [ + Finding(slot.Id, Severity.Warning, "Slot a is inactive.", "drop a"), + Finding(slot.Id, Severity.Critical, "Slot b is inactive.", "drop b"), + ]), + new CheckResult(xid, [Finding(xid.Id, Severity.Critical, "Table orders is old.", "vacuum orders")]), + new CheckResult(Check("other-slot"), [Finding("other-slot", Severity.Warning, "Other.", "fix")]), + ]); + + var lines = Render(report).Split('\n'); + + Assert.Equal( + ["CRITICAL replication-slot-inactive", "CRITICAL xid-wraparound", "WARNING other-slot", "WARNING replication-slot-inactive"], + lines.Where(l => l.Length > 0 && !l.StartsWith(' ') && (l.StartsWith("CRITICAL") || l.StartsWith("WARNING")))); + Assert.Equal("0 passed · 2 critical · 1 warning", lines[^2]); + } + + [Fact] + public void Says_how_many_passed_when_nothing_is_found() + { + var report = new ScanReport(Server, [new CheckResult(Check("a"), []), new CheckResult(Check("b"), [])]); + + Assert.Equal("pgcheckup · app on db.example.com · PostgreSQL 17.6\n\n2 passed\n", Render(report)); + } + + [Fact] + public void Pluralises_warnings() + { + var report = new ScanReport(Server, + [ + new CheckResult(Check("a"), [Finding("a", Severity.Warning, "A.", "fix")]), + new CheckResult(Check("b"), [Finding("b", Severity.Warning, "B.", "fix")]), + ]); + + Assert.EndsWith("0 passed · 2 warnings\n", Render(report)); + } + + [Fact] + public void Colours_the_severity_word_only_when_asked() + { + var report = new ScanReport(Server, [new CheckResult(Check("a"), [Finding("a", Severity.Warning, "A.", "fix")])]); + + Assert.DoesNotContain("\u001b", Render(report, color: false)); + var coloured = Render(report, color: true); + Assert.Contains("\u001b[1;33mWARNING\u001b[0m", coloured); + } + + [Theory] + [InlineData(false, null, null, true)] + [InlineData(true, null, null, false)] + [InlineData(false, "1", null, false)] + [InlineData(false, "", null, true)] + [InlineData(false, null, "dumb", false)] + public void Uses_colour_only_on_a_terminal_without_NO_COLOR(bool redirected, string? noColor, string? term, bool expected) + { + var environment = new Dictionary { ["NO_COLOR"] = noColor, ["TERM"] = term }; + + Assert.Equal(expected, TerminalReport.UseColor(redirected, environment)); + } +} diff --git a/tests/Pgcheckup.Tests/Postgres/InactiveSlotFixture.cs b/tests/Pgcheckup.Tests/Postgres/InactiveSlotFixture.cs new file mode 100644 index 0000000..3b19422 --- /dev/null +++ b/tests/Pgcheckup.Tests/Postgres/InactiveSlotFixture.cs @@ -0,0 +1,43 @@ +using Npgsql; + +namespace Pgcheckup.Tests.Postgres; + +// A Postgres whose inactive slot holds more WAL than replication-slot-inactive's default 1 GB, +// so a scan with default thresholds reports it. It starts on first use, so skipped tests cost nothing. +public sealed class InactiveSlotFixture : IAsyncDisposable +{ + private readonly Lazy> server = new(StartAsync); + + public Task ServerAsync() => server.Value; + + public async Task CheckupUrlAsync() + { + var checkup = (await ServerAsync()).Checkup; + return $"postgres://checkup:checkup@{checkup.Host}:{checkup.Port}/app"; + } + + public async ValueTask DisposeAsync() + { + if (server.IsValueCreated) + { + await (await server.Value).DisposeAsync(); + } + } + + // Not tied to the first test's cancellation, because every test in the class shares it. + private static async Task StartAsync() + { + var server = await PostgresServer.StartAsync(CancellationToken.None); + await server.ExecuteAsSuperuserAsync(CancellationToken.None, "SELECT pg_create_physical_replication_slot('debezium', true)"); + + // Logical messages add WAL without writing table data, which keeps this fast. + await using var connection = await server.OpenSuperuserAsync(CancellationToken.None); + for (var i = 0; i < 17; i++) + { + await using var command = new NpgsqlCommand("SELECT pg_logical_emit_message(false, 'pgcheckup', repeat('x', 64 * 1024 * 1024))", connection); + await command.ExecuteNonQueryAsync(); + } + + return server; + } +} From 33ffe4aba5f5796f9eaedc9439c9c73cf79c2813 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:14:48 -0500 Subject: [PATCH 12/26] test(cli): run the published binary against an inactive slot --- .../Pgcheckup.Tests/Cli/NativeBinaryTests.cs | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 tests/Pgcheckup.Tests/Cli/NativeBinaryTests.cs diff --git a/tests/Pgcheckup.Tests/Cli/NativeBinaryTests.cs b/tests/Pgcheckup.Tests/Cli/NativeBinaryTests.cs new file mode 100644 index 0000000..99fa4bb --- /dev/null +++ b/tests/Pgcheckup.Tests/Cli/NativeBinaryTests.cs @@ -0,0 +1,49 @@ +using System.Diagnostics; +using Pgcheckup.Tests.Postgres; + +namespace Pgcheckup.Tests.Cli; + +// M0's acceptance test: the published NativeAOT binary reports an inactive slot and exits 1 with +// --fail-on warning. CI's AOT job points PGCHECKUP_BINARY at the binary it just published. +public class NativeBinaryTests(InactiveSlotFixture postgres) : IClassFixture +{ + private const string NoBinary = "Set PGCHECKUP_BINARY to a published pgcheckup binary to run this test."; + + public static bool HasBinary => Binary.Length > 0; + + private static string Binary => Environment.GetEnvironmentVariable("PGCHECKUP_BINARY") ?? ""; + + private static async Task<(int ExitCode, string Output)> RunAsync(params string[] args) + { + var start = new ProcessStartInfo(Binary) { RedirectStandardOutput = true, RedirectStandardError = true }; + foreach (var arg in args) + { + start.ArgumentList.Add(arg); + } + + using var process = Process.Start(start)!; + var output = process.StandardOutput.ReadToEndAsync(TestContext.Current.CancellationToken); + var error = process.StandardError.ReadToEndAsync(TestContext.Current.CancellationToken); + await process.WaitForExitAsync(TestContext.Current.CancellationToken); + return (process.ExitCode, await output + await error); + } + + [Fact(Skip = NoBinary, SkipUnless = nameof(HasBinary))] + public async Task Reports_the_inactive_slot_and_exits_1_with_fail_on_warning() + { + var (exitCode, output) = await RunAsync("scan", await postgres.CheckupUrlAsync(), "--fail-on", "warning"); + + Assert.Equal(1, exitCode); + Assert.Contains("WARNING replication-slot-inactive", output); + Assert.Contains("Slot debezium has been inactive", output); + } + + [Fact(Skip = NoBinary, SkipUnless = nameof(HasBinary))] + public async Task Prints_its_version() + { + var (exitCode, output) = await RunAsync("--version"); + + Assert.Equal(0, exitCode); + Assert.NotEmpty(output.Trim()); + } +} From 029e331e0b2cd54fce7dbf8020c7da3a3991171c Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:16:00 -0500 Subject: [PATCH 13/26] ci: test on Postgres 14 to 18 and gate NativeAOT publish --- .github/workflows/ci.yml | 51 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 .github/workflows/ci.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..70dd6db --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,51 @@ +name: CI + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +concurrency: + group: ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + test: + name: Tests on Postgres ${{ matrix.postgres }} + runs-on: ubuntu-latest + timeout-minutes: 20 + strategy: + fail-fast: false + matrix: + postgres: ["14", "15", "16", "17", "18"] + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-dotnet@v6 + with: + dotnet-version: 10.0.x + - name: Build and test + run: dotnet test --project tests/Pgcheckup.Tests + env: + PGCHECKUP_TEST_POSTGRES: ${{ matrix.postgres }} + + native-aot: + name: NativeAOT (linux-x64) + runs-on: ubuntu-latest + timeout-minutes: 20 + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-dotnet@v6 + with: + dotnet-version: 10.0.x + - name: Install the native toolchain + run: sudo apt-get update && sudo apt-get install -y clang zlib1g-dev + # Trim and AOT warnings are errors, so this fails on any IL2xxx or IL3xxx warning. + - name: Publish + run: dotnet publish src/Pgcheckup -c Release -r linux-x64 -o out + - name: Scan an inactive slot with the published binary + run: dotnet test --project tests/Pgcheckup.Tests -- --filter-class Pgcheckup.Tests.Cli.NativeBinaryTests + env: + PGCHECKUP_BINARY: ${{ github.workspace }}/out/pgcheckup From 9bd7366c311c836eabea6675edcc0e78f786865f Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:22:51 -0500 Subject: [PATCH 14/26] docs(agents): add build, test and publish commands --- AGENTS.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f2eed26..9e7f5a2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,12 +6,16 @@ These are the working rules for agents in this repo. pgcheckup is a read-only CL - `README.md`: the pitch and the "safe to run on production" promises. - `ROADMAP.md`: decisions already made, the milestones, and what is out of scope. Check it before proposing features. Respect those decisions unless the owner reopens them. Record new or changed decisions there, with the date. -- The repo is still in planning. Don't build past the current milestone without asking. +- Work follows the milestones in `ROADMAP.md`. Don't build past the current milestone without asking. ## Commands -There is no code yet. Add the build, test and publish commands here when M0 lands. Keep them cross-platform (`dotnet`, `docker`), because the owner develops on Windows. Avoid bash-only scripts. +Keep commands cross-platform (`dotnet`, `docker`), because the owner develops on Windows. Avoid bash-only scripts. +- Build: `dotnet build pgcheckup.slnx`. A broken check folder fails the build with its file and line. +- Test: `dotnet test --project tests/Pgcheckup.Tests`. It needs Docker, and uses Postgres 18 unless `PGCHECKUP_TEST_POSTGRES` names another major (14 to 17). CI runs all five. +- Publish: `dotnet publish src/Pgcheckup -c Release -r win-x64 -o out` (`linux-x64` on Linux). Trim and AOT warnings fail it. +- Test the published binary: set `PGCHECKUP_BINARY` to it, then run `dotnet test --project tests/Pgcheckup.Tests -- --filter-class Pgcheckup.Tests.Cli.NativeBinaryTests`. - NativeAOT publish on this Windows machine fails with `'vswhere.exe' is not recognized` unless the VS Installer folder is on PATH. Run it as `$env:PATH = "C:\Program Files (x86)\Microsoft Visual Studio\Installer;$env:PATH"; dotnet publish …`. That is an environment problem, not an AOT warning. ## Safe to run on production (hard rules) From e524581ea622024755ebd0932e59f30d8a17fef5 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:22:51 -0500 Subject: [PATCH 15/26] docs(readme): add a recording of a scan --- README.md | 4 +++- docs/scan.gif | Bin 0 -> 45604 bytes 2 files changed, 3 insertions(+), 1 deletion(-) create mode 100644 docs/scan.gif diff --git a/README.md b/README.md index 95abfbe..9c5969c 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,9 @@ In February 2019, one of the Postgres shards behind Mailchimp's Mandrill [ran ou Most of these failures show up in the system catalogs weeks ahead: a table's transaction ID age, a replication slot nobody reads, WAL archiving that failed last night. Teams without a DBA rarely look. pgcheckup looks for them and tells you what to do. -> **Status:** planning. There is nothing to install yet. See [ROADMAP.md](ROADMAP.md). +> **Status:** early development. The first check, `replication-slot-inactive`, runs end to end. There is no release to install yet. See [ROADMAP.md](ROADMAP.md). + +![pgcheckup scanning a database whose inactive replication slot is holding 1.07 GB of WAL](docs/scan.gif) ## How it works diff --git a/docs/scan.gif b/docs/scan.gif new file mode 100644 index 0000000000000000000000000000000000000000..78a4d0295201de617134e6d32cb318e80dd30009 GIT binary patch literal 45604 zcmeF3S6oxux9=ArKnOt+nlvGV9(peuHS|!WH&H_`0sFL2>IxrZ_z`(%B2rx1-LLd-k z2puysGZYGCfzq+Cu)tsd3ExAE2}6o zsHgy{v_}n4Q&Uq{2h`OMsH>}MX#rYVTH4y$2M-=Rbm$P~5VsEP(M9U&>gws~>FWXd z27rNqfuR9lWMpKFWiU1tGd4ChF)=YU1Cd4H`?3V+t=6E|0KP?Kj8209}o}_7$_eY7BBE^%S@$vBqiQvS<#H6I7Nwg#F!D9|Y=C@7>oMM77vUM(&zE-48tDJdx{(HOIl@7Y=G z++D)Gdxi@Oo(l`ag@uKs#jC5UN7mL(Zfs;cf9~=8dCBt^(Jx*c+1gHf`?lcS+w_~)NjKYhyh{HgiN=fE#th`=r&yvt%?U~8kNV`-oyr$F;`01*BFW&qRvB(rxk z_Kr#1-ar25ko?af`TzBhKz0E-hyb>!xQ7VgK)Xv_!u40JL114?c$uevv`q)TY z^>meGnMsjJd(CXEW2@i&k@niTMqFRQ0n?7Ud#ygxHOEFf>KCsCt&J3!UTe62BmDJ~ z`=i$y|M)$Mo<-2CvuUM|#36lrth0G-C{4ulidk37#%P}M$p>Ryt&i>$>m>@BUvGOn zO}42$K7PIZ>0Fch=oRxD9nTj#Pd|Mye&gE9KPW^NA&c(L?KNtq^oiTuU9TTal$l<& zxOx5U)A`nuf84%#1IGD2V-oJMB;%5f(GGT7RS{J`_fO_>3%-&}u#Gp-gT=AS^P&%GnJbZ;D%9z`$ zSRlZ`tmKEqAxKOKj;l#35A2x~M4y4^MO$XJ+@ck&9oN$IujgRpfaQ!{MupB!OK3bS zgHfi+UlE@4=dj~?j!T9YgKC(-T92MWHwcEb(GP(kK(6t{X~zbo9(l3jP|0eSAuAIs z4Pz~IfD-t%(PfMm^P@tNINAy74ihm~uu-8*u+;gW(!dosU2-%B`pM&})Roc!xr!zV z99E@|N@FMqZSf_=5q8l}>n?JYfPfp0%Z!SL3nQVTAiIetwK`LN&zd6x+^`uOZ6!%p zSZ+=vQ~=1d5wU*0gl8Q~gE2$Ofv~k$`LZRxdg(SGy_b8mE&ciB`-9T*&ivZCqsq6+ zt)oGiS3styt15jg%r75SZFEB<#<&=*1c*Z{`+6)##0(%xR89wOXk)Ly*`!$oR9Zf? ziLuK!s7|n|uEqLN^i~?I$?|C0a8ZN*>v02NID$EX91I_4T9bJ*VgA@8B{yhm=pyL&!W_3sw^uRVXa7&IjNekpV= z;QjsZNA>R?oPWRj{QVzMOmaKR(fb2;R!9;JJF7|RFLu__jO0G7XE_9Z*vRv4`0%js z?28YNisR%yZkFc;etb->YWVo1?%IowPn(A1{(ROp7x?G%&PNS@zUY4c;?I{9CizcW z{rgXS+NMf0etI>g{_@l72_yN>Z)O}$eSSOd-T3+4(%F}v-!I3>f7w~jJ@w_oW>w>t zkI$~X{PO44ko@kaH*=?UKkqzh-2L+D{mb2509FhzVJRTKttg0DF=#)P!l<={hbI+- zC9pkE=Pd%ySTUnIwP#=C77+<6ff!+X*$cL!CCy5p4%A-m>sv7jNhL6EY#-mkR;p(ik$n9a8YRaMP?xYi{^<*!9;%TS-;gkBsj` zF1)_7Ggfuz{jkcNiyvQK1z5=#CX)#Q-=FCGt(#VP?qRiG(^J2yoMW0LY84ffR17?mx$lUV8XMs3#79p=|TgO7uwdV%Bx@ zaQ8tXe`CPOGm~Y5$s(q1+Qy0%7@4cb^XalS8KR7kR$*ZFx}0KDipzacUaRNO!M+3^ z(uBK6ch%lonR8XZn|y2?}f ziaCCCwoI?nFfp3aoozxx zADD1yh;!B&+asG80w&8Srt2M#AKYYJlF5cY9y2lM4zF>D{UUYZwg6u5;RFH<{(A{9 zv?bX8bqS_{T{<3Y6A~UoTLQDr03w9|XOnZ8-xAg*uuCX~hEF1OljqQ3{X14=)O0|p zU>#E2ljqO|o8=*gtQu=m6f|GiJuw)`P}78#oPiJ-RQun30`poEDT_l(Kc8zYjbC)65WKt?hv8w|HTd5qePFo!NLaktCV9^^{>Q?pb&?$0lcieVvb3HW? z$3KQqQi%ndx14ZFd2@Zz&1l42_H4}9gp3Jly(3;wmalLUJrGvq~j-$#GINinBtletjO{o?vx#tdp-j205G~c;X85F0*C5yFOgNV|c<7YTsq+ zzr1ahMyOCJy7Ey4PA6mTa7m37Y9kInDzsue{kBW2^YR#VHQIdx>{}{UuT-48CVKql z>8EaQ6Vn;$Ea}Cl=-K^}hOAzZARioKl67-{BD-zPcEiIYdPV6_W-XP)NgY%A@Vfz* zpUy2(dp^KE+SJ;1o^I_mFhKVILD?0CSP1A~{x;g~$L9X=&FEs`z0K|JsK0v{&mbb` zLf*Tn*%8(rn{d=1oQ*fezGrB5d1H0V!2L-XiD5#D6+vXELY@YaL^D^sK zhn`Q4{@*(^ria_1ec#4cht}*-kzJVeGu(6NkKqi=#2CeX@Mkb@#u~SaJEI8@;x2_v zclX10IcB1U6x+Ik7@pW6b6mU_1AWQ{o|xdZePT>XKod$@fkdc3;LX!u+bwv3dNd3f ztC$8$*U`zW>&+Coamb7J-nL=l0kIQ?%s8g5fMPx3J_pe20q%C|>f`m!IZVK=+vpY9 zG_~hZKUF}!KfHPEFK>2R*z;yqm!j|9EUcjg7qjTMw^|uU;^#gyrIF-5lnOeG3#mxY z8%~gcs?IQ9xDZq2oc2zvFXBG3pzOa zIEzlLL{TK=jm%7g6Z972TM~jUs(A!8i+ZT@+^$ObnE7%fHE=#j*@<<`CYefcW!{Ct z5YRactUV+fh}sDpWIj3rmvQq6F&;&z+89IbTxT$t20eb9;?N(PJM+zMoX}T?t_+y_$)OP+kmb|~Vf`d}iIBD( zM7d$g0eNnHT#$%C2C!*nqBvnDCIsl;dIO2FlI5X)zia|pUdLXE!`^dhPSY>HtVW+kA#HPqPLOjxLcAtIOZ}QNE3b$gUvwn-p54+ zhu#5SE}bjY18h8sOlHu|lFTt231{AVCzKpM966FT8kTdJAb%l*5##A_mvs3(be)c* z(9aDqC7fKzST-f@-AwhDmyGkoyejxC&hON*1A8}<3-0y9pLaU`#{wjNyP7F-0ehi~ zvEk0|d-sy3bo<_3&Tr&CDO~cmMkdJn+pEljnc(L|SO=jvfE%iGk0;5gyoNPUDMU zDW#333pm||wTR|NP<30Z_OEEyRI{^rf@=Rm`^tt%#i2iKp-1Phy z11H-@u?Afvx@pn%huBmI5RL;4TDBUCcC4ysxijkMG-Xf^o(mqUmJ$?~DXmcP)4g71 z9rMJtc31M5{XM+MgHn#m!N8b$yyjdlPqgbk9qs zyA2R{_Ny)N1ea1$!Fu8|*p@ZpEG9cIfv!ohF}%|5WGVBcH~_YBTnPb-ZX8y<91>I0 zk|AM~Nk0VsGZf-be}Gk;N@l46MRNK<|8NND>GJ`%t*$2k&f<1^3zz0%U9TUr3 zrW=7<;ze_~gfW_6mA#6Ye7)>zZXe~SNwR5lEGDq zzWP*bz`b96%7gq2Wk{f1*U>u&3^rk2>x-V{RJmamnl3M*+{;>456yVRMS~q9=vLK1 zUTMb?N0SBQ?>c#TEtfj!rz9l##?7VT1N^Mgp|4R_oH3zOExp;^X4;*-1p-GqR`0dB z4ZjF|%JsszOMd+LCk`1jph@XyGtYbF#;zg72n{~%9yFEB;6xZR&0Qbrigxel|I{f4 zsIOlS;5Wa%sGKTRy=*dY{Y;JhRq&xgbAG)4LOVU^Z!^1TW`0Y>a=yp5*uA4|2C<-i zGqV#ekQRp!LzPpvQ2j~ts{ZRf2oWltL-V-w2ZV=7hTxG>0ps)vGeG|2V3w5ihDn^P z@;h7KOo{P?-;O-2gS>fYnJ0ea&@3<4x^20gvYeeGJAS&%CDWNudvCw#zdKJtgw29iq9({Y7*L^BQ&19vY#o62e0|o#DbV1 z_-Y~}+pxDMmrY%FQ~%i{r_Lw^1J_uwmNUvGJh#Fqmtz6A%7I$C;@&zFA+X!_>>wdV zuZa|DE`d3e-lpv7dM;AXz_%jfNl!ss+S>dRjJ=OdK;WBDq^86r7W!Y+kVyFB~44>0jFE& z&pc#Gy!?ro90nG~T7$t{es{lldNa!_=G|ccMuGlFkyrK3dhnX?8nuM;RACJm)5D^v znP(A>flb6QbHF>tc~1{Yxfq8y0$aG^I|t!GyKrMNm$Nr9 zf=Aceti$@7y(cnEZDuHXeZDS4m)+lTsw#Zy;O@Poerg}1@Ub&YeKV_lj0{8x`X;2e zLzPvT#rOnkm3_5M3f9G3%-^|}TQB9wR^EprJ$d@6@6MCtE=!E0l^Qdt-{&~|`Lp2L zxT}_Lhs%#*tE0ci6A8KpkA#p-U*iciBM$TB%uXuq*Lbq^Bc5!2kJ#B?lL)Li{y#{C zv0xEpT5T6)BwFUs&VjC_wW zf7L$NjjzGm*GdM?a)KIIP!uX13$WtX0R$|lVe3WEfVG+vc zBe>W#O-UHAF0YBP6&+6GgFWYi^z3V>L7OV6W{rwL$@0<(brv`+Gt9_ITn?w6JWY-< zsnYGy@-hUiM61*J2W3*zbD1#BhVL9XUQJ|Ku#~L}XqC`7!lCr9`A1N-;fXPkulWa# z`Qs3c0bha$etDOS{Ao)c(F^Z&Kxw#!)&}~yxM!m+#y_Mc=VQNylHLfN|9ooVogwqM zd9cK#3#i6^xT9NSW2gV3ZMbw=`p|ig3uydnp#+!vXsG~S_NW{eUqDjBP26CX$(X%o z-nsQDCoY>r^#PW3?4pI`TKr9e2?t#I2r)Ifwj0MuatI*JvfuP7uEAr8H6qX5flHl} zUV)f6whoQ-MZI_;n#IS;we$>R!$n_c>^PrN=?KcGvn+9ug<7@i%Y~$S95p2K#1t4L z&J+X1JS$+CsA$MpADh!U`fh9b zsYoGlCp814QL2zEyNB&)>}V#Z%|2Vo(=;YL`)?1kq=H72#pzF!ib8?RI&OMm?+i1h zulW#9#%di8J0-{40Ag}%K{czE-R+wA%!d-*LS}~9L0G<*Q%anBT&$Vu6Z+#_i_V%b z#5gSacik?~&TH5cy^2UWN;UodZQDw?uym}(7W-bQe2QMZR@z&ul~yVjJ;^S)MC&6M zE?d&7sh=wexqo&J+waa%{j+nF_MjR!Vg0lQ#vqAAg|vN%q^+tQ~qUD}SmRl+I~9EG&1=Vh>Ks>xDdN^h~#tYk=An-w9-9#=%=&#%}4 z9<*-pGe5*eYr;Wv9_@P9;d15(OfLgojty<7uD2Y|4h*xaCPiW8 za1|jpM|zZB9$~m_7}G>TGM5>Igj+HM@IKCRjX(=(EZFbW_X%Kq7GGv&BBdrlhO zgR}zWGO=&fOPMvSH29F}IYx0jSO7M*%$m@v&k*$as01i9c7M7Yg^z*E+Zp8^VxNK|oI{$&M47g7Wo7d7 zU?eC}7!m#csE`k~=1%?_1Xi6+@<9~1C5v8jQDqZHTrrDAhN-t(nxc8UNF_3+_p>H+ z1Y>!AssbEoEBuYf-urec0mbi?@9m!=dw-L;86+iTAQ2SjyX1&i$E5S~YN_zD3md1> z2l$W7GLKS{q)b7bJ)^V5wp6mDf<}m&F^ouPo=~9Am1fpI-76YwVq=EV3(jzH>g6g= zlss|h*++=~WQ>?WA&p(K6fs#{JEZ?{j457k$1}rwX zbc4hvpEM$J+}tx=cQ1mgXE#l9@_A3)BG)Z;Op~vrk1F$(R@zs)jat~99Q~cv-P_eG zzM=k0408Kte>(ropL%Y7^C$W1uEZTn)Ib7Q9_?R}?%p2{EOckkS+DnIh~MaeJ2K@Z z=g94*FY%lfkJ9Q6L?KClj&2!L`PI5)W@RHEeNkJf(iT-r7UJz<{rX7t@JaJaAKA5_ zcT@28AzeC~^ts3)1AIGgot_Q_tJTEhNlxg?46Y|JCpVlC?+fqb5>PpC!w zhhk9Sk2x6|wdxbZPIAjy*OoXVKE0_vX_|=B`(Q#qCA4U>FRj{{o=g|_r23P)YqvIH zElLvJXqGcb*bSW&Q))m5^;Opur0(N(#9I_4T|UtOR-n1oT+_95j_BYsVh+_IPc7^c zVKnxAN#tMHH~z;l`U_Y52Y{xZKbyq*)Yg5uKlyjHsH`#<2aRp-nZMp(1Q4R~jZ1Vupv;B<5PZ^io@^Rw z30%8?z{DRabK#9_)($$o06e2%otuBsuuj7?=Up1t`SAK1*SUEP{OtbkY(JsKMA}0A zL-O>mT*pT>dUNjtGAN>FsLgBoct-ES8s(r~5p$Gr&Kpw8GOJ4zqy1)e zEW0aIblGHJi8?8gfaiZQMktS+wF0=*l0l~vvbn$zPqiXPdJMhRs1|d`Kw|1`3ySXN z;0iMm;nHF63Dj2J8bpv>vSc8+F#e#dns4?1}plHjOe`i2*d_4tJcw@kvTXX27l zg(AD@a3ghU^rv_XYB*g*?XM2#UcoM?_yIMIUICw9o_RJQzFQ4YqF+&XdzALaW?N26 zvK8Z}z?SZu(){R6q)7K^M@~-Tl+UNDmmNPE1JEVQwG9YIvktv%AzufCqs3F3GQxcj z(St#R$gB<`d^Hp&=y7>gV4-+_I{VPb3*d}PD6zH^L8|yr%NTfySV^ovn8sopGNzM7 zW3imPsBZUaFM`;mhQ(&kB*coUqE-_dy&>x_8D79sroWft8|R7->a;n&L%rC zk5Jsj`yJA)m{2^>jz=a*m=jO>pLA66hurUYe1GwIzGaeaPz9eEjgKAy389ZCA-CNI ze}3_rAK_)=pLmWRrgM)~BSZx2d?tly;YA|EV-q2&lP)5E5!|KdX$-K|wGpzb`YAaA zb|8#`S6-z~qlGyTrQ|)#SEA)hQ4q`HE#*87>d6D&&BL?l8W6Nz@pV>%^1+L5rEF_G zi#jX8EK%|Dj;Z1XeB}(l?SvE6w=(E)x(wWsGkgQ&)hQ_iUxs!Fisz-NM4d_Y!rL0d zwHPg?FYxuv)YQvX+byaQT_T241_oEv^7&qyX7W=4E}NfI(?r$kIEQD~TzmPT-4Aje z+pS@^Huy^s*8XG5roLU(nA)#b^E3Hc=RM<;F?SjqVfaXXP<@5Dd)6H-IxgKHwB9Fweft`4%9&lV` z$YfY_A^e%ai0To(Tj&p^GC_;Fi4P24SbLLVkJFl`d{RYw3G|n^S(m~(n@bbGy5Q*p zGRaWYytr)ThP&dRf6;Gghm9_ZmuQj<9ZXEHP7}a4WiSk<)gH5NsNvLLcHn{m%QL+w zH8RG&NDW3y$z^GMWT6k%A_OBc&h@1@Oe&hI*&^vypl{-P;WTYRrM__{aeFx%#h`r9 zH{!MN_?*I70lkK=8OgEM`E=$6+a~uaH&rNwmXQRD!1HdL#r$0p4gYZFd*5vZ?@0p8 zc;trv1si;CnI3Q5+;*oXiymgDNdm;U9pbIKCPV$0fE=#hKvC0UFp(1;0!7!W5fv61 z-72D*jKw3Zj2c{((U~lCX|6z_ek(kC%|PP@jUMoA@XnRWrp6v%DSqaZE|U6?-Sn>i zT<3{LEoaWDR~&F-P0*%S@Vv|b@i~2524#eY{swq_ikO%a1|{X{i09PUh*@d`#K7`JW;#$ZLc%j)2cIL>Vi8>XdyJ|a1;(n? z{KecHt>Mf$R6Xa|{fYqj>E=|kw=w)#(VtrnXKjqPWKgGHHyPAsyLe`?gl$(kbBJcV zxZK8im8s?f%Fvk|Qxmip@|5P=9D@r{I9i)YJG)LBo+*7K1+yHo@r!Ri_;G~$zOTT) z$H+Q89ooEGGkEmL+s*^r&K*7@s-7Oknfti30c0FESFV^ed#F1{m0=Nm#Djk9tE#@D zgYmb(Pz?D_yjc-?JVM213eTJDynZjF#-4CId@>*D28u!(l-M&Md#&m%>t6wR898ot zV8D4MSY_Cp{<`VhY79xOX&@13sX2Jws`=O;yguz|M}5o2$Vmi8Pj*e~BilNQmCr`k zLw&&@^V%=_f2txjf9&k)cR&6Q@fSoTunn>;5*5$in69D;-@wMfximCYP2q_-D5(a0 zOpK$1PMt>p09UFo$pdVCOg8A~({UlBhjpZ^p;|J%YCwixs2JOcTe>k??J`_*S;x;# zMyg0Jd&cb)mEKB+sliyD-(G2 zFB*KdU~kV<@nhnQpPMC}%Vi8~#FDLgXgQy2=5((XieOV`{rJH+bUy;mwx<#N^s7U^ z{;{1!|BS^t$A$Z8a+c$@J6o%?wjqDm8y|RyVTvdN!G)x>L8E@h3JT{l)<>gN4?bAp znSN-NbMVZUXiatiEW`(SG3Gm%UCb&Jw+eegTB>8WQhg-@1zTn_I~`%X*%|7uUwzaj zX9jKkTq0y~b>u#`oZVz%gh$1~s5+O7&Mt^W@Dp2)UN7;NJLL0^H=g~rQMin+kd_oc z_y%c7=Dm6(u$YC1528#bn!nMmd(&!dFkMiYpnVTPp;vD5Va`yTSl7=2KJHv!mvofm z2K=p`If~{YVcg5`T8_|-@f5!xb6yf2VqOBZ8Yxj#EuAg_t&9~UEW2aV4@^HRRcZnk zRy%4!MSG>KO@Rt58iz1t(fN%y{?98Yg83Tc^mV zZ(zi~Hhm(Gs>fZS+o5FFv_HLDj|6s%ZiEGZK><&HaY-i7UuDQ&Ez>CPZxS4Vyq|;3 zo&@LTGGwJ22j&eCG08@7DM-agTeA=(cr(DoIB2sa(Pafanc=RN2bj1JxODs2+|DGV zynz%m%IMGwMSm21D>lgA{t8wX^w`#+hz$V%(%gVkE7P&k0gXN``zs}CMeU>+qVNeN zU417tug2ai>4-utF7&JzEH<^FZsbuKKbywJt0I5v*RJI{SJ9fpgA*$TcwY4o z|BdZ3on+46E`l|K-Hlipfbm{spBz3G#L-Is&JsQ$_5OFuJqDr@BMG}|`jT{b-qzG?I35e4mV9CfX} z{b|6mSN=t;w|&iGr_D{JH;pulAp&0nQ+|fD0yLZdp`7@4aodys3~rm8?D`wG;R`Lh z6vY0U%4AyYKv2f+=q|lnl$cKu*cnc#*kWvsQ!jhiANhLQhl#%wH;?VBLK56 z3hna`B2B>HZT_X|kDCwPKI?g7roPDOv#hpG+5RK)(}gJ$G~AF1m9vHPUyTvX9uWGa zLQteF?Kd*C_&uio7vx+SDP@WUKA*S0$+_4Oi?$W}nzgxsx;oPIgpvfkdUlKOjQD8j z4HMYdYZL4=tz*nrknL7ns^ca);MtuC{wHn&u~RV(u-ObL72uKjM6M@LAZX555*4HdFzUHgDB zaBd(ASeP&|xDkeSNC6*O8t2CcKDhaFqxiQU_Wo5^e3x?7|0U&0?*FII2+`iBq8Th2 zC7Sx=S#^vt)g(>gH@6jo(Q$7324?_g-0Yr>OXLH#R!@#u7RvtSJUr{Q%;pqvC6@Cs z9N>2Lt-JNHOtpNp(UtBK&r!81jP#8e*t*WB#UdG0yK`NI7=t`v^@97tZzpF^4pS-z z;dNVewc2+3wpW5rKXr2Uf>w$ybUw{F7N`*NP!di;ikFsHhvtRgP7*Y!dxb?>PijTOAC{;80|Sf`cFAj6p;;d1TIQI3+aL*~N-L@0GzKUj zTL9BcFCEF2JeK2P?C(C3FYYdPK7YD$EERbMz0N!H$fD4|hjK|z_DmZb*jDS@Aug9$OU+Y#tI`BkgV$ct1XinVdjhMKzHb7nW_NJXfx|zmS)6~od(F3ZU;bx-RY#sF$Y*>eA|#_9@@!Kn zUdAC6Hrbo4{}h{B;vv=pM5Q{n${v15;N+xSJ?0rFZgx*9Hy5t!yb|;08}|&S9u>`k zY-y-#sSq9KiU@T$OGQEO<(X=0t6q zi;@6(C<{#&;lhjn2>1gBYdERBlZv=1a=(5TM--DxI2vK2Z2QOr42PTD%dULfpTLl4 z*=~~lkRoz4_{z-VCggmLo2Q!?R&;K#P2b_r7On>>ugRs2q(+>nNHr}B@}*aMVx0_k z{}b9R44Ep`i4$aw#5+D`1*@OzJ>zGkd?LY9_0%ms=g{~dT9mgiPceE_Am~w1K#O|y z+`t!F=6)3S^1;vah)6T{8=M~dZZ6=w_*KeUSn{t~;mTo_lq(1IxF752BnZfVIfv;_ z%6BO-W7l`G5{qJlqYIyio<^F-sxhH?241fZ76>w{As1dRTNN8K$;oQHt{xz(=kw7h zb(w9wT^Hj`r-Yg#wa_6o$y%#teT*j3&)v3l`kk)r2Ez3*;D?g-x`nIRu0bP6qjVC) zB3^aPt^D|b!?Aq#w(rztE586;t8aZ2u+Ht7vq`n9p$#)l`q4Mj@?XgD(O(=m<;Q-0 zkCOkcs_Q#ZaI*>97NIExWg|cFD0v!XN~kM|cCovU#)!zHXi?H@AQyPm>9;X5o0W&c zrx^EVArR$KzWcIpB{eZjmcb@5kCjVkMcVa{PgOwOd?D*tX+QJK`n~$Vxi2!j4qmql z5cl`W$i_bcCpRya(_|F1DCvBGruqA~Q8Mqx=6pYV#s4bR`b$Ldk2EBT$ZT!d)p zSN2j^Etzxsv2TJcvQspvQsGFvDCi(?V|U8dXye33NO$Ya~q%}+89vkz4H|0| zF%v4%Qe^KLoj?wn&R#hm?7P2^wGNlwQ+RiK7@k&cRXq{UhnQkxICmcIc_e|+_G5UI z`Bd9bp^EaK9g@o7k8>sPWy$p)&Ta=TmYSwo$c+RMHHF(u25z21+joXTAY;bq+7Z7< z1$utC`(Lv9{{#*zrY4A*3YuxWD%nqPms>%%SLk4KB!!^*%Nx9vhf-4Pnl=60O_KoI z9kRkib{U*0`Jp}(XHsEPR#0TpOeXZ-lfA5BkOKu^(Rl`}Y|9FA&U;MomZ% z38l91^kV(wIle>BAj@Tj>7q;pg`}1NadkhR?1ECIHRD9Jux%xxCug0ASU^jgqeyNg zJJ1GRO`7CMz((1!mzCGy6m(|c;i^f3<)~X|0auS_rSLN`6} z=z^=`A)7Q9M*de=X6Cq!G6eY4!#?F9Rt8TV zK%{|vt;-DnJHcVa=%`w`HHPt@wf=~jZwv+9ddc=N$6zeyZ{KYUN$lA1M>m2o)Pb+6`i1+NQ^&a8SiR>q*!r>|q&IG8Zf>N}TGQQK6?00a;scu5RA z2d-TZvjw*3EmH8ROK_dB$8^<3t?YwF>!0jHb@r&)UNWd&Eq!5`RspL$ zw}%6t7RN8EgYY+ryDUE;q_;oZ_y?i?PeC(L9d_+ug~I(wfcSjyTjoR&ZDIA&H$2nh ztyY&z7>HS!?nX&cT3yiQ5|WaUfzhih8c4%HI*SEo?qzthI>j1QEU|GMy{XG~khm3#T zD;ZYL7%4iRAdYHwIVZ_OLdiQTM{2DI;dT8iADc(;b6*ER3VHL;dm6QF4QUrMu&xVj z*84Q+7efEQej0$^R^!n|DmujBkS$kD3@e;(&I z*_JS^{Ck}Lu-38sALIOLtM=KRe~t4C-l>fKKF+U{y=eV?oZsJG?%O!OOpg4waekL+ z#c$*MbrVWo$N3qbMSLCSKPy}Rb)5f9OUl=Aem?KDy>b3i_ixcq{Y5^Fy>Wg$wT8WM z{_uP~+Bkot0iqXs%F7!k>TcydEvQ z^ZTtYy8t5z?(sVT!iEwh2bEA__X*2o-ZEK|5~vfn53(eFVR zB4fbLo#e?fqvvfqLv7VImH&U;2>(Yt;P3tXpCk6SF66(-mi!+-Vtd}f3p)HYS=pP2 zgHvb|asQid4~2O2WQzUz?Vr8@94AeN(aI%ij=fn#xo?c8q zZx3+ndAQz0at>=#xwGLYEH+yA(7|)D@kY85^A{bv{fv&PC^4T_Sv0J_(Gj2>rnj3# zGo_?^yp+x0=OHSsPu1>rrapKHXQ>y*?*Hq3}oSxwch2>QiUO$0MC=?>LophQjrkGi&eXMs`NAp=P zmGuxj9jhXwzuqG{GD9}h<)D@%3tRaR%sC+dfCsagDbRtfMyjH~4u_C*M=Q)Ndvcj4 z9dxrD#iKFV9NNg^zU)cN4-fA&2U*f{G|d9}yZ#>@9EO+;KfH>5?ptvg62PpJeO^#> zvxIBKNv8mEa6#PQ}#n<6Y`2nTf4$52B$ zT%GI*`tHk042br^I5PEzkl}j2laXgYBQOo6DUcIXuz{lCgJVZMjuUaG+x9l;@OWnEBai&{J_nE!DBR&LZ4@i!z%Ybzqai^J;lFCEIxFMP# zQ^6LQ*7%Fe5j{_JIkpzZBrPpMuPNOrES0Zu-$K3Sj4+Neljsj^(Gry^e zZL=HRRONRB=!;K{`EtYA2aFR`uD|}{R zMMtJM9@i<%+KC+q{;0y>hjC-cERN+1BMN$eNnks>w6v5`yo zJn}NJWK5DN&K1@l-oN}M80@m%%OZ)3R#JJNdhLKaT$qPmbsd8wnH96$neXGV&`B!N z!Eg*|L_<1|teWM684<;d$iXzI8xIOI1c}+tE?l~;O4FZ^28B8%;yuG3B4wx^^p#sM zrBB9?cI*Im#}?EfYZ>Vr9?LPf1w*y1@IYKwMIGS`o}wr$Wb#6A9EljSb!mPGcl=i zkaIpG8fVta{9w9YSxAA1TBhR-lhkDo36fBDac4i;30B;RD?7G@f}%;$@^#CSTG~Xm zqmThb+aNeNZ3-soQzh)!bL|r63Rh71a-z~yPpnG_FL$*rltYQ=(%BP3AjiOj@JVct zb@nnJJf=H}MP5LZbqJRRJ!A#-jalWq=fZeo7Xh`0JQs45ysblqw-eJa9Q9Zq3Fn?1 zr7wN(9>?N_`Ea;;uOhgLGEj7!j`^sTKFAE{#<@g4Qymt=a7^(H!<`)_cn#fc14-h!RLPi+yV4gN&G$~@-GkgGd5uL zE6^X<3@1u=uf>KB?Ay1>=pb6sR=rHGaCC*FDgr(%*wmxAUYw`ev7PL|KOkW;9dFsh zi!#_6Q1UTMCmHDREzA!f6f{^gN}W2m=JnC=DUF~*(af!#PYzovB$#MGIU1-e;uhOx zB#3Jmla7uoO;hS%8-l2(+}Aq~B*uK!1LYN;&<#sZJcAZxt`9d%i|;9x73c%WLSC}f zOdeK$|BkH@;Q6qSn6|%L_YMQO8zRYdY60w&SOiM z(%(&s8Bl0qPM$piAp(Z7Y;NmBWS4>Iv6dbl0w8QjTw_mvlC^7Pr)j&zE}53;fhtVPw@?g#!Y3tt*s;kWR0ASSgoGNJh9Ux@Cx9Yy)DsjvprVsXC}O~d1r0^T zh7nO?J)sHq*b&PKh>G9$2+*7+NC2QtdjS?2;_N-X*IM#CzXqhLtHav7M=oGEz zlk6j(Im+B0m7|4+rV#BvR1dO>L4tDg{4*6&+^a9YHm)Bp^ntMv9L zi=kRcm+KhviLbI1SyGp!R4gYN^u#t3B+3K4C@0a`z3flaTu$yp zr%zjyAyBLs&9qG3hp^Pp5M%a^_=TCDXiPSkaH^DLTzC{&35W#?@~7 zbDs_Sh%n*cz-o)UoJNC8q`_A!kfU112cKwlW;R^C7he$@aewPlvv%oY0(|~1QLA1% zwhEadgqyVZ0xn^$7>ArhL_8QbUSLKsooEFaAgwQ3az+1_-G_=@3qdy%&+@%b=564w ze6T~a$}EPC8s?&%puG_{0apBbCGtHuaEG1@)stPeVX=cqj2hlzkQHd~&7Ww&n%V_i zO35PVfG{Jtk{|lFrA;H^#-peQB+-+xPAqRJ51#uBrn8ZaVltPF#7rm$Er2NoouVf1 zL=h!Qvb0$r^mXNrQL;}KI92$kr;yx=A~J(Nv7UH=jV)Ew*=>V2Z^iYB{IjgGi1p-c zVuh0g-oy>9#8dt-H^MVM)0$Pw{z``-OLCJj$08NiAte6CoZIxJyqRbjZXh%1@xAQ1 zkdSm%YcHBMr>+4P!<`(eCa>4b%dJON@+e$1Xy0ege{h7$Vs4JGs^|XFZqcSnHTfUs zzwhivyoBW9Rn}8PL8Lq|i;L|}!+#sXGy>vPFN%ql9Ht}nlM`y9P-zGF*#g>l^V%A3DPw}>Rc^$4Ubx(wn<9I_L`vGGEDt! zT!faS=QWgMko6iugqm2EvCeS>8Z;1t)r8aNM0^T5MEDpnX+SUdYbs^A9=n`J6|)ua zF|bZJE7(N52Dq-ssx*r$EuAqNH53tmlcALU*`xtAF+xMQsU_OJke){Hl1s!vF>WfC z5FNAdhS)JiODQx>F4a)fMr@{$tT%2Imsj3ku{_)!l;T#pq?kv|6k#XgnL1m|c=V!TbGdMh4xwuB_W?qX z27lM+cxDt{&c;qb%c=zY%WTMpi~aQ<&E7~Zup)ON=#dB2m&+q#x!6LnV8%4)CYKPz z!wqOjpFhxowD1lsHXI@AP?=o5n=8al(`%4skgLIzs^K!ghV&lXriZCFi337%&Nmth z!1@>orwv3eZLYO`)l7Z+eIp^YrDE28YM&mvH5H8I5{5(JwEt)UT=I~Ha!w8HTuHiM zh^*7XvD2pMK2f4qd-T*`oftbNL9MdAOV;465`1e7wQCo7Hc zjk}Zg=wTTb9!4lZ8e}#?9prTqn=z^rlxrGN(1VK8hRAJNN~|?TsipKHq-|RGphv>9 zGt_jfMXy)MA4=+g@F0;-xS*lj5|gjEV^BSOMH75zHBxGz9%V1N)d=6`xh@lXIq8Wj zWbnSt!XaVBfGO|-ugTRCe?hwm&I4Df;Q`^?4m=oqoH78wl0|F?cdThS;)&wta*s80 z4~MJe)c2SwgmO(w*&+;nnF=4d0?kz+shad_!c0;y z`bJ_Zb_m%L16dr_c9orgC)hzF_Ec)q5nTv@L;-;`2UD%tQ**LeW zbPj-v(O`VV*r4sW7fehn8`qntY!za7LfkDcOBZg>G#&?HWB(>;gG|#-ZN|pyvF>7Q zj|X=7cWkT>+b;#ffo}Lc7$wHVpp|Ae?#2(A%eibPgDY2!2{uGJnXpHehcS7$^)4mB zT5P-+o0toRi*eRh3+(_L^yau(-ydL_MiXKag#9bcD-ag1KOE`iA#UN^kWls@yBXZ_ z23cLJskwS^O>Y0zyMt8@8^|T(b6bQF+HP`vPww{K^gI9XG?&SGeUlO1TMfqR2iE4| zY#vRV+ugDpT(Y>26^A=mi+ED7t@OUC@SbG-VY3mOu@)<9D?KU%gC%+HLQIsJtkcdv zD4Di>yA}N+(i~Jy#V|hJ-1qX)b>axy*r|t8Dy@l8>?*qrQ@do-vU$^;R(KOGh5}I*DQK?eF80>Yyut}#$;&- zA~x;ibYGk@JFZn-0%-p!bIH?!hymakPp)7{+1cjy1QE1GmKf5yE9^X?T^+$(CjSA727 zlE?R!{kkWfbbtAb`=#^lud29T)^vaM`TJ`h-(UCZK00YgI%7yaZ%A1&q-q+fK0j3Z zc&P5zkb2UCh8Yhw&wH@7;z486gB|A|?0WoQ_pb+tk@|jEBd6ozPZ1)XlRT);>IU?co?N{q&?qJ?9Zq`lE}JELuY!U2b}K@$sV( z9{$Rt$GXRl2B?p3RXjeF^0@!kqoK!-+gBZVZ0dddWX6+L>XZBICu2=db}xN0e&or! zrU!4D9{q2|zniZ;xtH?qSKkMpoBsVteenMAzrgjOACru@bwk+sM#yi7xZ6m1a^L*e zh+Mc2JB~1B-nW`R!uox0NIT+iePoRNudU2j*kGc#QIOu6#YNW4b_j$2Uj?U zymr{K{L(AAVyUHgzsJvA0ny!;k*>Cs>>9x0aCQf!g`dI{)=N`ZOFB2$CQM-UVV?u& zRsZmFg(C`9Vf%>kNXkeP=hw&HDu6;o`yzf{-Lx_C`S?w=;(uR{TUdY}J8#%6ew=!} z+WDkR_4h2(ZCW5vXtap1`02`{J$Q8dM#THWqdk$ry%#YKveM-gN1w$Ap7!}$Op-nA z@JBr@p-lfTLk0yXvjj;9`+Huu{$}9&c88grx(146FPp z?6nK>BVGxPDLsE|?gd4!jIA^q%upxI4g(o`=o=cJ*14w`^!MrCjpnD<@r<@;+TR$> zC@pXWgT65`pVya_TfetWkE151)=WzBb&XH#|7~?74aow!J@)q{Y=UmwA!X-B$L`Zl z73uFV^XpR%jh}gQ^7Xs3(bgtK$j3TYJQm}dcO&HSX;IybB=PF&5t%Z-`Miq>ouv z*%i0M;C!#P+(E5(7GX%NN8ir%yVjxpMgglfZ_sTy@9RcP3~o}2U(=GFK_cdvXmIil zqw|TpD2IS5`=q2KfiW&weUh^7`BqAd7`$5;N&EIa|vgB#ifyawl zE;;$1H^g&xe5}VN+UuGL)*D&NV-ln8_$Roe=PJ`IYhp%~WDK9%Ke<_uUNb6#{Q6v* zL@t#lF|)ggWXxsVE?-eT=%G;+VjJB^A_Oyhaa;juZE_|v6%L! z8$ut{in(^A|J-AH7nWI<%y`gN0OMG2U|mGCDp`+Tv*BktdB0D3yyJc}mL7LYnqg?M z>IlLA59b2qm-bj}1mY%*O>J;7`M5{IcGLWvC7Z`UoBiTg?gtcxmg9K zUMtoO$?xetZ?tD&tonu&MZ!^ejy|8EYsBHr)&^33a*M?d=pv)~Ih|CSXVYYefflOi z$#1CckOu3Q)lJSdIQz=seohMr_kU_GsW9~Usc!DBZwHv&Avnjl`Wn01F``b~?-5lG z>o&x|$I*VzvJw*gXY|TFU*&;C;PeNfqx5U2f+~!vEa$IYuP1_(eMLzh<<0P*t7>a3t; z$Lg$xs;6UF#K_fbd?eC>1hUcJYo*dzU!{;)Af4jptR{_twy``ueKVhU^I3N`txWHc zo7zmC&vmDpfISX3&D~^+Xv7zlcqsXJNf`joHSs#@n(>lw-k!m6EGbAM;ZEaWsjNP4 zeP{zigs}-0NNl%`OkO-Y-}c8hfXn9h7M*y8Zx$%bgT~qurx8&?=afH&`ND~jdV{nAs&wH5+b4oyt$Gy>j&A^t&I}$ z2Tkp6z6cvoq;v?<+&MYX8c+>=oHrp2u0{-kvFzbK5?)`iZ$;Beq={p5W$0p95XUhw zF1(G6FJtviH2E3ykLzHxd($5ZzKX2B-H z!NJN+Lx^RwM#R{Os34D2vTS#crK_aRbL-_wQSt|;W$IW)+}}60T$S$OJc#!5__EPv z?d%*(o^;X&F_h1>a35%wo%vuS*AZ4R%MleJ$DbtOXy|8IRZpse@%*oIkeyZ#B?`ha zHZK-(j=8+UVJdUDe0D-R-a+^Q)6mgJHCy*d(lE*cnV%S~qJ!Rkd3(Cr^^|grjQ09~ zB9}YvWXI}V7uO>mNyM;N;A-dQxWZCj_3yGC`a2{Z z`6`vK))!d6)Dk%s>_$*iFv;(0T)+;wQVK0-d=5qP=`Ba|b!@Za8D!uHLG*BObu<0t z_<;QxS@y&-7{(Svg;N=i& zGbEo}Vx^L%=M(4PbWn%te}Y0SWr|T}eZ=%>MQQ+n_IDR!EPR1ynB~4nyIgGg;|RX5 zLJ2up_BF2G^Z|Da4GH6o4d>WWBDcG`%qi?<&9c%4;!Km&`vx?Ba7E<1n;T2DAlPwr zAjCqbs`!b?hfN>C;xyHzWd)8!!1L`@wuBAZ2af$_9q&gcf394x0wUdvjT}X-|E+U* z)bNexTgR{2y}t)xtSxPuobos++WR)g7UvV@yWJCM-ETkil09<*9Ci52T{A~2$GiNQ zGwTSwLqE;CdXWT~=y#npM!`WYl=3yomZyIyHRn~%p>SW{j3b@+x2b`;Ytq|fhlEoH zzAsH~d&k%bSxq`5l#)VvtRoP?*>BhHl-)S%Ua$Ki*rkSp2LYQ~zbmhozgwZP|M{_f zRz>q>RJEqImw6J2k94a?*K_jzFzdU+pN~v@*`HVZzDlP*DxrN?Ip#FW6c=^@0aWg{ z9j^;{u0Q=BLc%Kh-{V7(n&w3j4;@}`-UDWD_A$^>uHTc-PdFhID*i63nv4u^p{3nW7gBE4;&nX2v(30jlUD}$Vbpp|HjwTUJL;F$)1u4Ycrkea#a@d)MP z=atA;8a*rAw;m%?6X{&gM?BB78(J9%CRxwyLWy)XxQYkUGNo@9fIFVksodIP6rL~k z`CO0LD=Mp$m*u&@djXua2Gb#0FHyj1A;>Zoyl_W8oFrNs>s(WP&6?l{07OvMN-m|@ zK<3<3ZDdt#;)JAzG)xe1lWtO1bEzN1gd7RsgP!Uya``E=FdKn|+)W;4bjn+UMWBgx zN<{m~^D=8(XO7ao7+t^h&~CE<2eje~z#^jfe?_!!8rlaQ$(sjt>E;rKF?Ts_pkn%4Z zkFg~58>tV})F*n%3b8%)GRdq4Rcfli037~En+K4uo8Wc&X*r$@CpHwfjUK8YNpOE+ zZW51Z2$ejzp1Yf4iBa<4(_C^!Ah-#I%RiD&Xkn3I)-<)_ejd>orB5?LN4QW17rHs9 zlCt5`8q&P)v|Ln2}&JFIl#HQ}tRgv7%_xRJM*a}3`q zg!N_-fe{E6qf3`+RxM+YbUe5cvC1@%U)>~kPAE!furnLF^%;<9i6sSuRxWuik0?Wf zd+^X{BN^4tNw^H15UJm}!R2G*E+br_-DfsnYD}c8$w)9tI?jf9cAD$o4;zN*Km38< z#6?)+VvaL#Votd-&)Pi|sBC9px&bC1SnD5f=O81cpI(8&+RH~mCz!U!I*k2RA9Jj} z3|plrj`-IzKYXWu+~IdKOpBYV^-1-DQoYEBw3uUB_@NfTI&?d;VEsZ0Iwf&bYvYZO zyaAiJUS;ozr}b5Yxt$(sIyqOI)7e1Xh;^0(!GXssImdDJmJ7{!mKLc%@ER4f+1mBJ zwXs@N3KE-59iD5t1UtJDe=sw|Do4xXrA#2u`$X#8<3aY;p=XbWud$5$VaY$+C9*%6 zKmFu_xhD(PoGjXTviR)DB@a(7`*BhX+fGT-k`2JOY6ewws(st3*vwOZ*VsVZQzg@N z9VOUqEh!Xx#9CApsbe~8+e;RSrx@DoFJ{TADGQ68l{GqfQp1+xa4!I@{y?)AlZJ%U zn_|*>ZzRY_XwSA<2EeMzbVL&@d0`c7qU<+FV)%sPyfQ&Ordg}{YozIxfsoJ)l(KQR z=@^%>`El=o%XI2}JuHxrZkfpN9jhr~idM8?x;c-K&O1=zPu_uw7>&fcJn3|2+z}u= zuzURwN(~#Mb4=vgD*EZOXA}Gh*TfWecdIBN`3A6I{1D-an0i-8y`!NX(4Y17$2J)# zeQe^>5d2vU^{$q(12`Y%49?fW)U{-89<+mpaaY52TyUm=^yC9=s+c?|+BNBb)1lW1 z)?&yqmUz}cwl~pedhBcsUZ)`i*WxcAJA*)C7x%0;2u>*Zrw$#pQrE`k5p;&Q?z!ao z2{ts;C*eIXz$T{q_&ywxM>}JWa^VafKGBEa%Y_HEaEI{p=bZlFH}p;|givhu8%X2c z!PbU7p?q9#wv`uxpJ^78blN%}38qU88KeX8{^0)!i9$Hg!73D`Xw`&GLaNv>Yq=U; zYs97V2%RSK7dy-aBe@tL-4POh#o$jEp#mD>dIiok zkd9^Jx>3rvE2<$bzI<1>;Up2`kH3s;AZq}onjDvFm4Q$kldYEVN~eHeqnd0d?Vqxi z@l|kcpp|m%6*GLoO!Cpi_>(Shb63{d9alqRuuUs4Se!SwD8>pBIj!Wvb?4|VTzLCI zW+6hEkK#|DvVeN%0)YK|jC4dlXB7%hU4>h1YgNjlgsPcMT=-K4UW;rE)8KZ7TbhS zuV^WkwXod=pjb1Z;Zmx&b!N@M>4EseUK9N_W#fcajgWQX!#;V04&x5Jh7`|1SO9dC zM`<>$TYo3=ESo4o!pCAk=FnO0S{#^!bP9XIN!v zN!@CAl|hoAC9Scgw3-Z%A@8_`puaop{T|cEqh1rjU^xBe2}&22@GTK{SV$_@!}~Rl z%mA*HM_Dz;hx*I#VIlDXm%L0(wpu!*Ixu5wt)=06)|JxjvB{CUWns&vyxfMyL$fpM>v^m`Q9#sHwt0D`cdC3xO;TJ1)Lh zL?{-M?rSdvi3kf2N~5XQwK>*1U8XbT7n&$@^~9}i(0i)YthQ}RF{M?@a}+^mA!5Cl z^6D+F*9a{}iGyllTnGK96=esn{fZd7)Dly^fLN!Yq`ea^%ZEOFA+@+-_VXw*BX+&n zL>VMl*>mxQMo6!{)b{XMr4hTv;2ZlC*T*FocZ`Da|}$qm~lGJDQ3g`lEufJA7%Mki1Gm;#hkGp|Bnx z#A--){Ms>=q$8pMzh?YRBc8btU!jIwov{l{G0mcZxkmUnKuFe;JA|1&MzUm;-~ex# znTII=2)zdKauK0`dxoa|;Qha?x2h?7*u=FwN;!|us)dV%xIJn(svU?oQCf{}yG4d6 z5^$!7P>E0$qjVZTm~fL81JDg4UVe@q#f81M5KkN7M`i^OjG9(zNms=Gax9;Fz6X}7 z&#W>`g#ESm5&!#CM=wSw9Vk9!5?N9WmaEUKRFeVAB{w6bcq&t_G5)(XXMsgT;COK= zURto++PPf=O4oZvY>+}FbZ-XEDWiy|y?n^m?C3DO&~>aEIlka+c^hSBB1zgZt@ZPp zq0W@4@O55L>#U}f2m=2W4~d%`KV6#bp9MZ-)Ww}&hqupsGjuFd8hdEri{5V1{=k*K z?Svr)M0Gg!fzRE$O_cuoF86k>bEmvg6 z8OM5)_&A!|&vgFb*_UgP-=`0Db!&Fz%rryA2No(d+eP+keSQqY*u@?#s2eHQ=x1uZ zRwry`w@YkxOQp;Ae%+5I(nsUcwLN>AqiW`%jNh%juu4wc5p|bchP&s4r-jIzS)+IS zU~|ydfHBF@#2*9Ixfi)bG9g-jO*)UYogHTF@^g_oy-eV;mFCQZwX`1yKs3L{E? zBFB;6e5|9cOmTtm4x5Cvp82EFT%Wp(%^`~%oQtYOr9X(+_`+`*DP8%pE%GQ-n!n8! z-od74l?_Kh;CZ^swwZ~^`L?0S0rG7OFTH5%Rm#BvriaB2XGRLUQKMQ&@f3jf8#nv< zEpYjqUw3ONrdg$ z?Z3!%qK5WE4e86-0w|d6{JbI1SajVR-Ym(Zdj)z*>z6*2;VrC1mTrfR7+nq(Y>Xs4 zXeX`%aNCN)NZp!~Pr`~UGv^O2WqMJ1^8Y>dy9{q!Kc-jYc!>BWCsM?AIUC|i_G$!a zl6OwLnTsVFx7_9~Gn*RbOn6?IRkbuIXUPv@aALVkPgwZdQ@W#Y1Dyg(+XrJeNbU5| zTjxEmXw!u4KGsXJ75wm0q=#J^zxtow%dzmOqR?%WIdRc%H`kgi^B8K#&LLI1FNE7I zoas3}{5vEpYE$m=^5-tnU7CgW*4v@ZuTF2Ne|oNxka|-?_-<0*aJP!$aIHy_&6>#E z^7|Yd!p^sO<;G2wqz2_JRnO__09hez!#K6~yoB3_eCrONcy2{|T$wC_e_P`lW}ZyS z&7G4sv-NG)FQ**VgJbLg=Q)xw!bjTV($UfzdIjR5s0l;lL|xJC$Z%nG5~I($iJoK` zvk)P%M$NW`0t{S@W4c_;U+$yt?_-O}$&3Y-gHN{{wZO>dj7ivTgZ+J!o)ag2wNlS& z?lHw9M<$)iM{o}SKYI5Or3#-Lqwi;4Ug;+r>$dU6B%S5zD|6qpFj(IQ7~=O;K|U_5 zJy}=guuI6kr}LPiq)R^vaWk|?!35l~kLh=;Qo6*Mm1^QmPHXn>kY80-#*AfgHPRq< z@5%}M;J|Oosx-aErAs`Q--XovQMC~17y|i^1V`K`Pgz~<{pX-BHpy+H;dN;9ZS@G8 zb9Iz;Lp_)8rmkG%_Qv_hcn*JzS2MGEX>hk%PraUrTl|shR;Z8lnDFn4D)dO7IV)yj z+Y%-0+koZiB0@l1{WkrU7RCCPF`TSc(!#M8=JBkH0qjM)@(faDciF{0uTzOHE238X zF_LRgf2|TS##~utbN%8#Rah6sM!hr2SA(x78tSHEI;d@ZIz)Bo@9`t!LF6Nf7dZ|zyc@XOQSMc?_~EqX4P%A1)JqPuUV*W|Dx zrkR9hg=<Gfx|{> z7E!QPlYfydF|lf3xDck;`YBzaWQ z;&%Ln->LYoiiOPL!EBetfYb=1)MYslHw*?%tf1;GzT~mR`*{^s+S&YZQ{8iHL6c%WTg^&7c88Eoxws=2ZgDesto`-ot-52Wchv zl8(4Sa27~v-QZFpTp45cE55+~zN9*?q~CL5=LcW!ayZZ7&%zt8BpIc*4SByW&vkOa zxWtSgx5T}X(Jl9nfBtkFf!>H<`{+J;%&|L*-xQo;CQCE6=3M+4Z!uqTczqGuW)C%~ z=${kNtn9IyuhHh#bH* z>M#1tFR3jMmYiugx9q~FU)DYum|FMozGv+1)#ep;eMko~5_yLWk9>dsVE(0EAET8o z`{IE0@c=6P1P`X%X<=_;DK`DCugv5KLA$c<;rYUv^uy$d%yYtSt)-&ckzP_eVXi&Bv$Zocs1EoX$ByjcLc zmr1Ui4|pCh*}DlJ8soiS+l|UYL?l`I~ z`?Q@r3hnDRLB#hG@9#V{4TtSC3;Nx7Z(mK_Q`prO zCjjYo8%d3=%H6U{&>GgRSaB)nuT=2mkLp-`xD=JS-EJSn)Wn7;(u9&g2%o88I+rPO z5L~oQF3LK%zOiF_5$aS1u6=ll5$=}oKt3_h7tB*4j2%fP1(PEg3zZgVWel|h?(vpt zF~gVq%>>j5fMI=W;AH>E3T114) zZ^Te|5;{A|%#oOBs3Juxp=*IUBPeQ@Gu4t*jpFwY;;t3k!sQ++wRX>%jL?9NuI z0m~uuQG<+%fZzjkBey-(#7F@M(*zO@s+foFvfpna-_elXt~q=qO`2jeM5)ySU7PBcm zx&P=_+#hT#qtPRG@ciS~GX%r3g?g!t0h7~O@NP45vGX+>j9ovVaoxn2k|N5 z^V`K2){9Tn4)*~NRoyWE+`l(B;*%F2g{>F$Dj|e{Z%iz zR|p|{9G4stq-_K&4d~_S%bJfseB&M&AX_wqVQ3}u#^p|9Gjay)k7=bvsEmPvxkI>f z2Qk_0SeOTnUv@t#!Zh(Dwi-o}{s2azKbmE+xmlVcRwO;d1g8U}Au!VvB-H{-Sh7GL zNmGw>&N!|uy?Z#&?NJy$yG)S|;9|WSob(Fw(KC=jh=z&qS&fQRbeB{piPh|#BaEl_ z$%_Z^aYgt)^yIlPKD%8Z(&(e?m9Z?vA|c_zf6b2`;VJ(y*gWOZq0St1<}HCdyAi)s zqx4I|Rtr)85PXK6;!W<+MU8-?PH6;)&(}b~EW#AFYDcygMI$R1!Y8Y6e*hQOH%iim zI(=PhkBY!-qas?QC>m0PKWutBhR^3>uQf~4)c9OZ;{g}2`l@P1 zw`&y4HxMx1Dk|!3&NeATVhm?0**QcB1x!Q;ZcZ6K+ko3nK3S;l)*7+djfx_Ja-K<1 zxklx$AyCRBOCc+9ksV)+&(Y2}#ymme z{>?#<785;$^-)9GcIl)Ugay|Wmrgg&VG{=rqKtC9%L?hw1F}MPgSH(F7vs&@5Fx*D zJ7I>h5Xs15;qu-24bj(yWhdWPUDX>1DSFfe@%;4NvgsH=3!}koV!n2A4sx+34eNA1 z@Zra7H6LK3*swuesEh|r9h8LuJCC-5(=;--5R8v{l2i)@ z56XfXb#GKsFQX)|@#xQT)J-D`H(_F}CWaNU;o65`n$s>1Fw-^DL$%llRpc!WCR8o+ z{6Vy5$p|LYmJbH2|JkmU_-fr+9%0&g0dF;SiWc;0pJvaUK;L3Qs3ce;_53<Y|l8 z$$`m(*f6d1Y9Q&}-g)E%>Ak(!&~|Ku9&^7J^|R;g`=q*X~0U+yeAR~9yeY5i5{ z+W45|tr}{2JZxR#gfuvXNMBL|iPtw~_(PM?KO8yp`r#@YCYk3lXt?Xbw=HHKG zk~nADI+cw{+KI1W3S}-uD2*laQju{XQeO>%*GU3-n+eAGXZvL{hQNoZ;M8_0@8^TG zpl7peHI=+>b@sEhe=hX6BGQ#TOW$3U9k=jx=d+xB3!^VTo9F0!Oa1Kit7mHHIfam8 z?NC^Dz%Sdc5U&R&=p{uet7i{REtt7*PEg?=_nt3({(RZz=YJ7ih?y@+JYFmhT9nR| zM6eWTLteNUFaB1(DBJp?eBX=JCts|&_+st7Me1)rm`)zA#zs7TA@LZO293+&#^ssg zio$W_s_{zYxN7UTxoY2d^~v#?i{rKT#y314ulqc{k?>N@TukLf{KN(Yx})Reuh|oD4CTzM#pU-+)hemuQspAyYw~50qgoNcDK5*D zg$;(sMCI=#)BuTHl@mjIrYoJFpQo4MUrl?)6w7~3hf;P(1m1Fsy0{ z)8oHjYs|T2xU~6ijm&rDGo^Fd&r*UI)&lV2Zg4lGIRcLyEzazY=WjK*bUHyI4z zl}sIHt?reU0dze&vq<5_mN1~H_#yN@MKaI?Fw`g_kRYNB@85BNl}cv%U-0`S>q z;NfE6RH@uJ`q%wvfXV}{HLs3|lNnzD<`ZsuMmQA$DQxiFb*KIm38T@p;2Zk;8&Kra z5EKK_On}7=BXl2l$VbUM5EA^4hPYyUY2r>kc^{Y`DA{m(UF5=67Oa0!B%iSUj%HknqJQ^S~y4-P{gl4=Ib=GnS~7Pc2B(%9O=GW$NHmR*^#F^tnjn)6yp|&|GhVVCxY1 z!ZN}f&e42XN0G|oVRO>!RAs6rWm1K*NT3+?1s_~hWriqjtId5#14&PtB$M;?2HC5Z%+GsR@G^j{1p)L?X z7o>6;#9La{gt=Xsm^%>-4v`W0B3|Px4duLARwTf(fpg&RO46Xjj-{BR$2fikKk+4k zA%(9%%2z2J;s9@r^1@*-rwnhIxX*4Jmm@Z=&G^@8NRn-khs#hqtt3g0|NKj`?;8-u zQ+gqo&@94XS{2zKHy1s`WNEkmBfu^hmwAdOx+iG|7q_4hNB;`8>^$i8r^1yZP2gU% zmxfJx>Q539rfb1`4VFEqNbLp$A%tmqOy-bcp9D3omm^=nKMeiq#i*yA5Zx%9*vji8 zqUt7NP7@(f#&1iXBDF@!8FEM|DLuRyl7_B&oA^PJs`bTrf2_*fU%js|2+0g9ERt^?l!NdyYYl#E!YuFwYc2Z`SNbvR?mHpo|PnDALE=ILl~p4Xnf`goaI{N(p}ln zX$zA5vKBLHPaXSXY&aT`Z26OS;WB?~uhAtl_`j{U81FTjIRn&_td*n@3uft|H)+u( zqIU!?z4ld%R^lV5Gux)^`?aSvbGtcZ`!gk*;CLfp*O}k5x;c9UnxTNKpPP~x=D`Lc zKXou_I5OOrPvj%*&evcZxV)BFFX|O0!(Y94=bH=O(Cdcb8vY5lT5eupnWq zM$k%S1BpehUoK1io$vPDV9j znPaiDiS>(~M2;U=g!tB$HfB)c89|eqRt}d>-d+Cc^yEFIj`df9?&r2r{q1Z&@)n%1 zZM)=uGXrGNu24SGDC(?F8?8FazRV);`FbCibc1z^U=`(9x|8C2iJKTWByxTASEeW( zeGy>*pIxEvq=%bNcUsyg+ZMpQ<7M;s;y|2T8TBa`_QsH#8eXdR)PY2#knf@#APHNKQIrOfKD7jTC z<9#k)m3WWN9Bjk<_eXui_lI-LZY8xrcQa%g0=F>k@SIMcOQ}T`TghMXgU@xstoX8H zSDcUK7x4M3t=hbgmQ5R2$=9EZ>2d89MQ`xo7`DXuUbXv(@kKMTNZu!Gqu`V4^*wJR zW7c-o2HgsG@mT--ccq_%@ch=hvss+mpC8_2r8V1qYk6x!IEqU)ICnA}MgkU@W$5T% zd~S6wj_>2&&vY`+vOH1xleB}Z3J3o?DhRb{@O0|x%&5_I?sB*xdiwd0x%7_vfX^+F z)vwi1lt=R~Y-!6`o0p7Fw49L9^7J)qoGYiBlqQrWsAKcn=bSHcKj_}bjtlg7W^>$K zLDC6g==-<|PV$RjqzwHZ;r;RTrfclkA<7>ah;6U)JKx&dSDvm@Y;WTjK|6Ky)F5}8 zh2;#X*}2iq)b*{Sp~d`IdkvCbo}{0VI2}f-fzTpj{M1Giu*tO zu2;}uJBKi$W<7ms*5L~rMycRwz;+uJr?99q{azol_) zl9lP&GMjQ2i8?(as`|j(N&F=f4G(F=I^?^4A$hbW9!i*Z=IJE< zJ_C6wvH%k-1mD-%G}06;?49F%J1vykh{;lGdF?JJ_3OrN0bYXp#05Gf(^14d zJsb;)bQjsxyvjUzY5uxh-fXu|Sblgx1k{f62U6u>!d{E6mMhdckYsPtH<7e3 z^BHrVB@C<(R1-z1)Z)?t`#b6m^~{l)Z}L|IxYK+;ZQirfKQ(p%?kX~&gi;(>fUJJV zbCWt#^Os;+2%g<8TO}I`%9^9*0vD$y-X!MoMuX=9#JMyVO!T0w?ObKjTG9vTk8=aaS>*ke`R{8N z8Cs}>Zq&|HTjSf9h`HZB)t}G_7o`?k4(gNv;?gN0OI-bYfs1a*o;IYmJezAqh5g9~ z&!$}2i`j25$8o=NYcfMN5pRT)@`c*w!tpNl^MZaS@v#lLf)&GaA;Pp4|H1V|E;eb~ zaej6KvZTT7thX6A*{htX3u;}m@ZFdxEzQ(9x^nE^h?NIn7un1q8Y|N+irdn?)MeAT zF^|a8e|V@d-4i0RPh&Sl#Fp9K5n&g^@|XIUr0Zf~yzkJl>U9=2lyf4&l*WM_R# zo`D=y7%6$Sn$*Q3co}{~MyRF3Wo&$Ybtb8Bw{)gW493T_=Axwm({}*P^fdTj%)29G zmnPSZmn8HA7*U&nD=5D5pKXT!=os5`BCI=yzm%<)^8Wpf573`Jn-Gt&zAEbayMD&i zY4M0!Y1QX)se$U)q^%l~W8WMBO5~_BRyK(i z%m;RBDA$c%r$wXZ6x3Uyq$LRLB;X_hao4NxjS>f4_N59ubF`lvG7bLA6SH7yhSa)-)P8V5^GBS{6M{@s30v_ z&(cYzd`iG}@xu~jK3a!)SM{Dn`AJ&9-zRFB11|9rCSy`x_`RpOjgpPl7qKy53Zx_? z_xRs=7`a*R$jAz3wkBU+z<1+wDLoW`@2A6bJmPugVsLU)`1>k07mJ+iW4@8vWbh!> z#8pL5Wz)M*yKrnS$sG~&GB=}8-&hk*?>)FlJws2(LT#&d?+(ZM^(f{r3F)hOoLWrG z^?pjY3_3r{FJx2vswhvvLvN+~JoTmaYYB5N_v3UqQP)#^i=y0awG~8o@W((ubqd|s zz5J`2>$zAg%N^(8ZgaxiN9~H*6MMxm!?iI%w(M>{AqFhlE3s+2A9+>JRP!y|Bv$N6 z7Eb`%A_=Wcn)Mq?R!hjq61xoW$~1u1$hU(eEg7g4m%qG@|FrZW!%aeMl$4T(nQh$+ z^(ZZwZz1B_x$%KaYHm{{GAOZkNczM9EZaa^r2FM;o@K4XE~a}YhesXc|B_1`^q_62 z9b79}J(GX?*G0us+}opYl5zzqW^?nuHnm`0M*Veg;sIFR}bBv(ZcK(eb5@F{CB?txOKI zb~EUpli0U+s|k)w!~d_lbN@;*-4{5#fFhRwMM5P-MMXixYgSIafY;PgbDNbpsAQy8 zT2@xh2zbBbB`wX&)U3?0yku?NY2k}0mQ7kV*<~73GA(Dq#wn*P4riae*52p*bbdQ$ z?Z4n%&+~oO_x(QW`F=iMshL<>7y^=&!;4;*;@#vFuGECgGo)cvfgrSdc3(HotzG5a zuKRP?if&ytC~W%tqJGn}prGWLFms>qxH;aEgte?(jFNQqN!+5h?3jmfR5HsnJ|~<1 zs2sdF23f0R?6Tw0Y6&Yp#AXB%KlERXlUa7jc7F*rZU?xqEY|qeL?y(b%4}792gc0y zE(v>yzbXxIN|UW_fH5=xM-Evok67Up3@_-HWa*R-&#`{zbPwEV9`XnVuqANz62Mi- zR*vT)yZbq#e2!d356(6I1aU$Y&Y=VLy@8}kh$B>RqQ<<@JSI$wa2f|zwGRD(L|8`y zj!J07!1U2!fD_tcT{dNYaF!>LvX=PVcIc^4Lc<|$kpvtyw9hXHeRcE95H`-0Q;f#&9+X^W(nZ*LUgXVNqZtk!}ruMcq1TV336pC0QG296U7>nUl<)!{^zNin?VCt}AT*A(H;w|L4G0SN_S+seH$r8Y77X>BD zn_}UF<$jm679$^pO|~2<=5x7m$+xrsi`Y*{gE+zdn07ul#m_L^$E!uoL?Tumfjj&B zge3n)nULos_a8-4>D*VlE>v~ay z=@V3YqURk!^Xy&ta8x&BO&(x$Lr1guEbebSC5$DS$wUe+x$xXmI21Ky<6i==sG#*1 zL5Esdel*`K%Y?O*hsl8K%7QK(;xqFL=tRXaRX?k?;uf0~JmPoYO-eXZ{}@Jwa+6^&mq=$YB<<{d~qMm5dob zeCn0Ng=!V(oU;9WhXo5pf9~g?YsKmXj9)%4CBg06>{)C+FPm={Y@w&pLiWlQU#J%bc-<%>qqS+0F@bGAOMmoHmJ z5W9fHl)IZE^KFyndCig%)et2agBb;EiTxx-KU2=XWB@pd7}})N`g<>j!9ZCE5{s6MtukkW`4b(C> zw!qa*!4=9bN4ZBnkVR^x^k0!T#@n-H2Q77r`omuTBdCpVBQGj(1X}+| zMEfQ;Y;S0|$9Y~;$A)h}jC)W6|4;7;n4FlcH@cbE{p>(3-~7?e{aav;%eJ~WA2!N1 zmCIa6QkDw6c11X%`@qq*!{L<@76?7O53WdAr7K!FBE~8u;Op0&`5;8p{4;$7+8+gC zJq>6`#9v2XPVDp~b`}jcMQ;bP?WErkk(f$ZRHSFj>POLDat~?$E*f$me}{QlIv>8f z+Ac6$EK!;q3?Bi{{?g-n93UzsI5mh*t68anE|8^c;$|%K!M!z~u(Nm)JB!{cR^9rb zISJG(3mlSVCz7;PHz!a4QCb(&(|1g0Gb-D%EHG6=7namCZ;70BH({?11Mhul^Xfc$ z8PPg~1xGTKa`r@t(b+YD3O-X9j-ThdUqsU6pz#t^doKr*%xAjE5nV9n(7kc45NnVw6(`7j2LOa9|0g4)_)WYITNMvo{)XJN2{w6LF~Oc4Os<)ir?2b zXC#;-vrgnU`7tG^GcyRfz&xO@kxV*0V%V$c`!;{~-M?lnBi;#Fl9ZQcf4uD;Oj~pU$GPU8gQ#Q(x z%VHlQtaqw`}*H!#e8S$;rXg?q?$=w@qR*fK}&X8FD#*EfO*uj&NcgU z>Mvh@tjw$qKdG#CzGD+3?A2K2<}{v;PwB0;Ol^nkw(Nd3UdV{G!fk8lO1I~ajdDy5 zI0vO1xbl!1Fkltv<2*7(U8AHGcy`~ZVtJFd)L*&th#YW{d!f- z){w*gFk7;POWb5NBjf11+|Aq$YpO_oUYtevdRkq%a)q{2M|U3S0TU)NE+KDSi!|~}abRM9uJWDcr z*5p=y<57>6S_mf>bxKF{Ajvp9jSyg2;1=F!y8bD-;_27^xDnq|Pj}u#Uy{Ev5xo~y zP%)Pqa`GPMZq;l`Otf6EKo(Z}o{l(F765$PtX $@pr2q2wcZNmaU2IJ?Kns$_T? z4s-r7N{6WbHZfhwTd`GL{_D+#lfIAue3iLzOx>L6HaAmqx@|7O==f<Ea`bo_c0A zZE(Be%gBZcs1xO<@=xxO=;^;|(q4|O5;k{J-~8}=vjiPo@+aJtTh{@+?HJ}fpT9P? zar>{Ilf@|O5mAcFK+uw!)xka7&3k9IKiy=y*1119x9^Xaa71HLPx7zDe^E*`s~WSn z0pOq}HMQBI*{WRRJdLOx<`8x)js-gXoYwXH`(symjs3aR-i6jCDP^d@<;CZzx4)c- z+^hWg<1bP|%+e@HT>q??oUVdTHza(GM~x87Vm{5C+Xu$I>3Co+m%#q1tz8M*4*L;S z@efA^Eg+OGk^K){tJMIYhr}^MB@%x?-?5Mu*x58(!6e0vD;KPlmE8EdwSkW;C1*TW zqPvgKMfcBvS(tO-(1~=ucwUuXp=E7A$9*x`# z6GmUqOuy-0Emz;Wdp79s$=lc)okylM$4{4ZBiq0WLEYS7_^D50!iz!CrKx>!^F9Ic z_E-08zr1+sa&^qpZ6@?MNTghwdt`l2ea3ZP!CLP(j8tl9jS4p2eZVdvywvbnYTeZJ zITN23-a&t>V69AVtZP4P+iGxA-?ty;aqpVlx3EpoYl(MyC$(9+`Dr%C2Cx4xM2n?; zv(F^1=lm`o7<~KZuc)X8w{`gFXU_w0yFYsoNSJ5J@1<1S;_LcoOiCQyhBPo)%yEL% z4}fT|*@mcfKfB4{P4E<0fk8&|R0M!di7T>upC&JKTVV{s%<=7E@YP2mrc1ql)+s8! ziPTeoe1#Ix#dIs>~(JW$lb4WZu!fvPDLM@dsQXOwV$aLozl7G#TPHn)Fj`S zn>mv*X#cu4P3M+Bn?7^-bzR1Xx!30;dJePo|8Qiq%r+K#yqP^;=I1ciR2i8#ccD6= zW$t3_p*M4v>T@05G&h|}d~><^V#}MB)*El${LntAb$Hv_^;6>8E8R0KZ?E=#c=NVR zuE&~hAD{#!BFv*kV*@e45-EzXcnL&}eT{(Y8P(4ST+a%H56PWAf?}Gf=I{XL;Bpt7 z^Qwm}@{&%LP(D>J*qo(?j{6BZS=_r1;+C=pxf0o;e9oxVJe zv`(C(Xms5uww(FMz~V~Nx<|+;`Y1SP;54AHNnEy8a_lSXhpB-5s6P9?s&Vk`bIS@N)56b$@j1ows}K6owj8X%#e}6_(|>qzD)MU*Btljo2sR-WQu< z&>}eF?sBZYqr5qIGeYYH{vmLPc<7I+m=8x{uaXxC$b&xCXzF(Xn^C`OHN;@fX3eMp zClJl>alF}VnPsh9rV{}dVOWk+In|L!Qv}>0eKjsLM5_^b`Gb}(Mhk^FKE0FbLlVcL zR)_8twWZ{MJ`$_|=L5VnRNAav0Hi{xju#8SHVWmL^(v#Yx|i3B!YrxUa^-5YIb%aA zLh1>pYZ1JL-DZerPl%-J9%YWe5Z~s5bR*3<7L~*|r^O%Dj`F!G z)vahDpFVTonRinzEDD4&wK^K@`wxV5Z6IL|&3T*hT3{4$gUQU*jL!zsC=FasjDM_`N8yOf`;;ghriY4;?x zEJZ<$p2Y0<8jo!g>$%Iq@>B*sdS~f+?v)goR6J!eL?7ffbrf5pMYwDA)3+1M?+_)!AaDCU$viJBotZ>HAB>zJvp-v+Q+k=O4`S^%FIS zU&>>*)>PjySJU-9C6k4rCvyVq^ho##5Y?Bg++fj|6U2r{Ycx5gi+>XEH0wG;`NYPt z=3)m%cC+7O+whZ^p^NL?Thf-{?9M!0Ar9V(32yUG@$sNel`JImn~`l zyLvX}kT$hYSUri_twOQWP9wLV8C~hiWdbxR2#|uwqlJ;5 z*L!FOkzWU zSr-hk>Y-D_cOGT-L~H8Rr9E^zdESC+Z0IyR!*enXtOn#oxM}UPP7T- zZ74QoiQi_P1d@c(BH_rC5u2J1s8pC3axgY9Ez7N@p&$r3-QqeXw{dI1@6k~U?eS=X zwF@|_3>Eo7x6UoxUKC#o5qrdU^pOBES!`+Puku~3g(G4j_;jM&hb!yACle7SiA&qf z^QA+X&HWHU3L3rvbD6zR7SInG-XP`}u1B6bDTY(J%Ccl?Fw;fDv(uMJtV z$aw84m)95aULd|16u5cWoiG`XYi$aLTc^Np2ShxvG9BfUH~s;4fmso@6ORgNONckl z-NmPr9((bYtKN|_{agCbx^=V7KO`-Ed6!8TI`Mf$x;*dv-E61cUkDFacl6B{diF0* zS(dVfzs*{>+37W7)V6frKjwX1o7b#;ChLC1%g7pw<=3k$K0PeGwb1q;V}8xIOFy0e z{zDIt_uJvypZk7)qm~oCk`~YY@UPVl74`&1|NC^1|C*$uv5?3B645K4C{g4QO{|uy z8cWU;QJtXOFj7%Yz7c9;?8>W6!za|XpjRXyQZgp54%gZnok8ZAQ}#NnsmXX@uq`xbZ|xIRCA+uQUgVcGF< zZ25)<374O?SEo%sU9slf-N|c5=ikcp*Kru9KD>L@R=TQDB{p|Gx!oNU@}C9c|LL5Af0vK5bmtKN_j9-o246N?BEb8zv@mA0 zGBg&-C0g)B%+y9f5!sp3DlVV)DJ8kbkPSLxhl?4$n200$t?yOY$0y?cWeC{6%f9^o zT*p77wFgG}&uGDP(Fz|=RkT|QJn>&cbUC8&PNCX%Imqz0nxRtjj=+ftlG?wV5mHAR z4%-=U%H}VP$#eS?f-5|B#N69zba7*y%Xc4w*Hk8iHu@i*BG)Y122>g;q-Qo5M-OJ2 UmJpxbJ<0lSgS7wG`?Bso0JI`uV*mgE literal 0 HcmV?d00001 From 6ba5e917c48a171dda02fa461a7c30dc4981589d Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 01:22:52 -0500 Subject: [PATCH 16/26] docs(roadmap): tick finished M0 items --- ROADMAP.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 5fb1f8d..1813e7e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -35,14 +35,14 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab ## M0: Placeholder (as soon as possible) - [x] Add `LICENSE` (Apache-2.0). -- [ ] Scaffold the .NET 10 solution: a `pgcheckup` console app with NativeAOT on, and a test project. `pgcheckup --version` and `pgcheckup list` work. -- [ ] Check contract: the `checks//` layout, frontmatter validation at build time, and embedding in the binary. One real check end to end: `replication-slot-inactive`. -- [ ] Fixture runner on Testcontainers: for every check, `fires.sql` must produce a finding and `healthy.sql` must not. -- [ ] Read-only guard: the fixture runner runs each check as a role that has only `pg_monitor`, inside a `READ ONLY` transaction. Positive control: a test check that writes must fail. -- [ ] `pgcheckup scan` with terminal output and exit codes 0, 1 and 2. +- [x] Scaffold the .NET 10 solution: a `pgcheckup` console app with NativeAOT on, and a test project. `pgcheckup --version` and `pgcheckup list` work. +- [x] Check contract: the `checks//` layout, frontmatter validation at build time, and embedding in the binary. One real check end to end: `replication-slot-inactive`. +- [x] Fixture runner on Testcontainers: for every check, `fires.sql` must produce a finding and `healthy.sql` must not. +- [x] Read-only guard: the fixture runner runs each check as a role that has only `pg_monitor`, inside a `READ ONLY` transaction. Positive control: a test check that writes must fail. +- [x] `pgcheckup scan` with terminal output and exit codes 0, 1 and 2. - [ ] CI on every PR: build, fixture tests on Postgres 14 to 18, and a NativeAOT publish that fails on any IL2xxx or IL3xxx warning. - [x] Brand: the logo in `docs/brand/`, with `-dark` variants. Then replace the README heading with a `` lockup. -- [ ] Add a recording or screenshot of a scan to the README. +- [x] Add a recording or screenshot of a scan to the README. **Done when:** CI is green, the NativeAOT binary reports an inactive replication slot on a Testcontainers Postgres and exits 1 with `--fail-on warning`, and a test PR that adds a writing check fails the read-only guard. From e422776a0f44fa8d0fff77e15d1c1fd41e0cacfa Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:00:37 -0500 Subject: [PATCH 17/26] fix(engine): pin search_path so planted functions can't shadow built-ins --- AGENTS.md | 4 ++-- ROADMAP.md | 2 +- src/Pgcheckup/Engine/ReadOnlySession.cs | 4 ++++ .../Engine/ReadOnlySessionTests.cs | 19 +++++++++++++++++-- 4 files changed, 24 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9e7f5a2..92a4fa5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,7 +22,7 @@ Keep commands cross-platform (`dotnet`, `docker`), because the owner develops on The product is only as good as these rules. Never break them, not even in debug modes or dev tooling. -- **Read-only, always.** A check is one `SELECT` against catalogs and statistics views. No DDL or DML, and no functions with side effects: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_*`, `pg_drop_*`, `pg_advisory_*`, `nextval`, `setval`, `set_config`, `txid_current`. Every statement pgcheckup sends runs inside `BEGIN READ ONLY` with `SET LOCAL statement_timeout` and `lock_timeout`, then rolls back. Never set anything for the whole session, because behind a transaction pooler it reaches the app's connections. Never weaken or bypass these guards. +- **Read-only, always.** A check is one `SELECT` against catalogs and statistics views. No DDL or DML, and no functions with side effects: `pg_terminate_backend`, `pg_cancel_backend`, `pg_reload_conf`, `pg_stat_reset*`, `pg_switch_wal`, `pg_create_*`, `pg_drop_*`, `pg_advisory_*`, `nextval`, `setval`, `set_config`, `txid_current`. Every statement pgcheckup sends runs inside `BEGIN READ ONLY` with `SET LOCAL statement_timeout`, `lock_timeout` and `search_path = pg_catalog, pg_temp`, then rolls back. Never set anything for the whole session, because behind a transaction pooler it reaches the app's connections. Never weaken or bypass these guards. - **Fixes are text.** pgcheckup prints fix SQL and never executes it. - **Least privilege.** No check needs more than `pg_monitor`. Never require superuser or `rds_superuser`. If the role lacks a privilege, the check is skipped with the reason. It is never an error. - **No network beyond the Postgres connection.** No telemetry, update checks, crash reporting or remote lookups. Data such as end-of-life dates ships inside the release. @@ -33,7 +33,7 @@ The product is only as good as these rules. Never break them, not even in debug ## Checks - One folder per check: `checks//check.sql`, `check.md`, `fixtures/fires.sql` and `fixtures/healthy.sql`. The shape is in the `ROADMAP.md` decisions. -- `check.sql` is one read-only query that returns values, never prose. The wording lives in the `message` and `fix` templates in `check.md`. Thresholds come in as `@name` parameters and are never hard-coded. +- `check.sql` is one read-only query that returns values, never prose. The wording lives in the `message` and `fix` templates in `check.md`. Thresholds come in as `@name` parameters and are never hard-coded. The search path is `pg_catalog` only, so qualify anything in another schema. - Compute ages and durations in SQL from the server's `now()`, not the client's clock. - `check.md` has **What breaks**, **Fix** and **Seen in** sections. Every check has at least one **Seen in** link to a public incident or the Postgres docs. Never cite anything a reader can't open. - Both fixtures are required. `fires.sql` is the positive control, so a check without one isn't done. A fixture may lower a threshold (`-- threshold name = value`) when the real condition can't be reproduced at scale. diff --git a/ROADMAP.md b/ROADMAP.md index 1813e7e..046bd58 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -6,7 +6,7 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab - **Name.** The idea started as "Postgres Doctor". `pgdoctor` is already an existing Go project, so this one is pgcheckup. PostgresAI's `postgres-checkup` is unrelated, and the README says so. - **Position.** Production readiness for teams without a DBA. It has fewer checks than pg-healthcheck or pgdoctor, and each one maps to a failure that causes outages and is explained in plain words with its fix. It is not a DBA toolkit. -- **Read-only by construction.** Every statement pgcheckup sends runs in its own transaction: `BEGIN READ ONLY`, `SET LOCAL statement_timeout = '5s'`, `SET LOCAL lock_timeout = '1s'`, the query, then `ROLLBACK`. Only `application_name = pgcheckup` is set for the whole session. Behind a transaction-mode pooler such as PgBouncer, a session-level `SET` would reach the app's next transaction, and the `options` connection parameter is rejected or dropped. The lock timeout stops a scan from queueing behind a migration's lock and then blocking the traffic behind it. pgcheckup prints fixes and never runs them. +- **Read-only by construction.** Every statement pgcheckup sends runs in its own transaction: `BEGIN READ ONLY`, `SET LOCAL statement_timeout = '5s'`, `SET LOCAL lock_timeout = '1s'`, `SET LOCAL search_path = pg_catalog, pg_temp`, the query, then `ROLLBACK`. The pinned search path stops a function planted in another schema from shadowing a built-in and running as the scanning role. Only `application_name = pgcheckup` is set for the whole session. Behind a transaction-mode pooler such as PgBouncer, a session-level `SET` would reach the app's next transaction, and the `options` connection parameter is rejected or dropped. The lock timeout stops a scan from queueing behind a migration's lock and then blocking the traffic behind it. pgcheckup prints fixes and never runs them. - **Least privilege.** No check needs more than `pg_monitor`. Each check declares what it needs, and if the role doesn't have it, the check is skipped with the reason. `pgcheckup grant` prints the SQL for a checkup role, including `ALTER ROLE … SET default_transaction_read_only = on`, so the role is read-only outside pgcheckup too. - **SQL only.** pgcheckup talks only to Postgres. Provider settings that SQL can see (such as `rds.force_ssl`) are in scope. Checks that need a cloud API (RDS backups, deletion protection, encryption at rest) are not. - **Managed providers.** pgcheckup detects RDS/Aurora, Cloud SQL, Azure Database for PostgreSQL, Supabase and Neon from SQL. A check can list providers where it doesn't apply or can't run, and it shows as skipped there, with the reason. diff --git a/src/Pgcheckup/Engine/ReadOnlySession.cs b/src/Pgcheckup/Engine/ReadOnlySession.cs index 6be0aa2..29dc775 100644 --- a/src/Pgcheckup/Engine/ReadOnlySession.cs +++ b/src/Pgcheckup/Engine/ReadOnlySession.cs @@ -14,6 +14,10 @@ public sealed class ReadOnlySession : IAsyncDisposable // Stops a scan from queueing behind a migration's lock and blocking the traffic behind it. "SET LOCAL lock_timeout = '1s'", + + // A function or operator planted in another schema can't shadow a built-in and run as + // the scanning role (CVE-2018-1058). pg_temp goes last so temporary tables can't either. + "SET LOCAL search_path = pg_catalog, pg_temp", ]; private readonly NpgsqlDataSource dataSource; diff --git a/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs b/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs index a1262ef..b5776f3 100644 --- a/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs +++ b/tests/Pgcheckup.Tests/Engine/ReadOnlySessionTests.cs @@ -30,6 +30,21 @@ SELECT current_setting('transaction_read_only') AS read_only, Assert.Equal("pgcheckup", row["application_name"]); } + [Fact] + public async Task Resolves_names_in_pg_catalog_before_anything_a_user_planted() + { + // An exact argument-type match in public would otherwise beat pg_catalog's polymorphic + // quote_literal(anyelement), and run as the scanning role (CVE-2018-1058). + await postgres.Server.ExecuteAsSuperuserAsync( + Cancel, + "CREATE FUNCTION public.quote_literal(name) RETURNS text LANGUAGE sql AS $$ SELECT 'planted' $$"); + + var row = Assert.Single(await QueryAsync("SELECT quote_literal('x'::name) AS quoted, current_setting('search_path') AS search_path")); + + Assert.Equal("'x'", row["quoted"]); + Assert.Equal("pg_catalog, pg_temp", row["search_path"]); + } + [Fact] public async Task Ends_the_transaction_after_each_query() { @@ -66,7 +81,7 @@ public async Task Rejects_a_write_even_when_the_role_may_write() await postgres.Server.ExecuteAsSuperuserAsync(Cancel, "CREATE TABLE written (n int)", "GRANT INSERT ON written TO checkup"); var error = await Assert.ThrowsAsync(() => - QueryAsync("WITH w AS (INSERT INTO written VALUES (1) RETURNING n) SELECT n FROM w")); + QueryAsync("WITH w AS (INSERT INTO public.written VALUES (1) RETURNING n) SELECT n FROM w")); Assert.Equal(PostgresErrorCodes.ReadOnlySqlTransaction, error.SqlState); } @@ -90,7 +105,7 @@ public async Task Gives_up_on_a_lock_instead_of_queueing_behind_it() await lockTable.ExecuteNonQueryAsync(Cancel); } - var error = await Assert.ThrowsAsync(() => QueryAsync("SELECT count(*) AS n FROM migrating")); + var error = await Assert.ThrowsAsync(() => QueryAsync("SELECT count(*) AS n FROM public.migrating")); Assert.Equal(PostgresErrorCodes.LockNotAvailable, error.SqlState); } From c2b753cd915f7d9a2ce9d67be7d4612f7e0f057e Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:00:38 -0500 Subject: [PATCH 18/26] fix(checks): deny functions that run SQL from a string or write WAL --- ROADMAP.md | 2 +- src/Pgcheckup.Checks.Generator/CheckSql.cs | 15 +++++++++++++-- tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs | 19 +++++++++++++++++++ 3 files changed, 33 insertions(+), 3 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 046bd58..00ff44c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -14,7 +14,7 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab - `check.sql`: one read-only statement, starting with `SELECT` or `WITH`, that returns a row per finding: `subject`, an optional `severity` that overrides the default, and the named values the templates use; - `check.md`: frontmatter (id, title, category, default severity, minimum Postgres version, required privileges, providers to skip, thresholds, and the `message` and `fix` templates) and a body with **What breaks**, **Fix** and **Seen in** (links to public incidents or the Postgres docs); - `fixtures/fires.sql` and `fixtures/healthy.sql`: setup scripts. The check must report on the first and stay quiet on the second. - A source generator compiles the checks into the binary, so nothing is parsed at runtime. The build fails on invalid frontmatter or templates, a missing section or fixture, an unused or unknown threshold, and SQL that calls a function with side effects that a `READ ONLY` transaction still allows (`pg_terminate_backend`, `pg_cancel_backend`, advisory locks, `txid_current` and others). Npgsql's SQL rewriting is off, so the server rejects a second statement. + A source generator compiles the checks into the binary, so nothing is parsed at runtime. The build fails on invalid frontmatter or templates, a missing section or fixture, an unused or unknown threshold, and SQL that calls a function with side effects that a `READ ONLY` transaction still allows (`pg_terminate_backend`, `pg_cancel_backend`, advisory locks, `txid_current` and others), or that runs SQL passed in as a string (`query_to_xml`, `ts_stat`). Npgsql's SQL rewriting is off, so the server rejects a second statement. - **Messages are templates**, so numbers read the same in every check. `{name}` inserts a value, and intervals print as "3 days". `{name:count}` and `{name:bytes}` print as "1.61 billion" and "48 GB". A `[…]` section is left out when a value in it is NULL, so one template covers older Postgres versions. `check.sql` quotes names for fix SQL with `quote_ident` and `quote_literal`. JSON output gets the raw values. - **Thresholds** live in each check's frontmatter, with defaults in Postgres units (`1GB`, `30min`, `1h`). `check.sql` reads them as `@name` parameters, which become `$1`, `$2`… at build time. A fixture can lower one with a `-- threshold name = value` line when the real condition can't be reproduced at full scale (wraparound, for example). Overrides from a config file come in v0.2. - **Ages and durations** are computed in SQL from the server's clock, so a skewed client clock can't change a finding. diff --git a/src/Pgcheckup.Checks.Generator/CheckSql.cs b/src/Pgcheckup.Checks.Generator/CheckSql.cs index d508d7e..98da3ed 100644 --- a/src/Pgcheckup.Checks.Generator/CheckSql.cs +++ b/src/Pgcheckup.Checks.Generator/CheckSql.cs @@ -24,9 +24,11 @@ public SqlResult(string sql, IReadOnlyList parameters, IReadOnlyList DeniedFunctions = new(StringComparer.Ordinal) { + "ts_stat", "ts_rewrite", "loread", "lowrite", "pg_logical_emit_message", "pg_stat_statements_reset", "nextval", "setval", "set_config", "txid_current", "pg_current_xact_id", "pg_terminate_backend", "pg_cancel_backend", "pg_reload_conf", "pg_rotate_logfile", "pg_switch_wal", "pg_promote", "pg_notify", "pg_export_snapshot", @@ -41,6 +43,7 @@ public static class CheckSql [ "pg_stat_reset", "pg_create_", "pg_drop_", "pg_copy_", "pg_advisory_", "pg_try_advisory_", "pg_file_", "pg_wal_replay_", "pg_replication_origin_", "dblink", "lo_", + "query_to_xml", "table_to_xml", "cursor_to_xml", "schema_to_xml", "database_to_xml", ]; public static SqlResult Compile(string sql, IReadOnlyCollection thresholds) @@ -75,11 +78,19 @@ public static SqlResult Compile(string sql, IReadOnlyCollection threshol var name = token.Text.ToLowerInvariant(); if (DeniedFunctions.Contains(name) || DeniedPrefixes.Any(p => name.StartsWith(p, StringComparison.Ordinal))) { - errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), $"check.sql calls {name}(), which has side effects that a READ ONLY transaction doesn't stop.")); + errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), $"check.sql calls {name}(), which can have side effects or run SQL of its own that a READ ONLY transaction doesn't stop.")); } break; + // U&"..." spells an identifier with escapes, which would hide a denied name. + case TokenKind.Other when token.Text == "&" && i > 0 && i + 1 < tokens.Count + && tokens[i - 1] is { Kind: TokenKind.Word, Text: "U" or "u" } unicode + && unicode.Start + 1 == token.Start + && tokens[i + 1].Kind == TokenKind.QuotedIdentifier: + errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), "check.sql uses a U&\"...\" identifier. Write the name plainly.")); + break; + case TokenKind.PositionalParameter: errors.Add(new SourceError(SqlTokenizer.LineOf(sql, token.Start), $"check.sql uses {token.Text}. Read thresholds as @name parameters.")); break; diff --git a/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs b/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs index 368552b..92b59f0 100644 --- a/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs +++ b/tests/Pgcheckup.Tests/Checks/CheckSqlTests.cs @@ -103,11 +103,30 @@ public void Rejects_more_than_one_statement(string sql) [InlineData("SELECT pg_read_file('postgresql.conf')")] [InlineData("SELECT pg_switch_wal()")] [InlineData("SELECT pg_notify('c', 'x')")] + [InlineData("SELECT pg_logical_emit_message(false, 'x', 'y')")] + [InlineData("SELECT public.pg_stat_statements_reset()")] + [InlineData("SELECT loread(0, 1)")] + [InlineData("SELECT query_to_xml('SELECT pg_terminate_backend(1)', false, false, '')")] + [InlineData("SELECT query_to_xmlschema('SELECT 1', false, false, '')")] + [InlineData("SELECT table_to_xml('orders', false, false, '')")] + [InlineData("SELECT cursor_to_xml('c', 1, false, false, '')")] + [InlineData("SELECT schema_to_xml('public', false, false, '')")] + [InlineData("SELECT database_to_xml(false, false, '')")] + [InlineData("SELECT ts_stat('SELECT 1')")] + [InlineData("SELECT ts_rewrite('a'::tsquery, 'SELECT 1')")] public void Rejects_functions_with_side_effects(string sql) { Assert.Contains(Compile(sql).Errors, e => e.Line == 1 && e.Message.Contains("side effects")); } + [Theory] + [InlineData("SELECT U&\"\0070g_terminate_backend\"(1)")] + [InlineData("SELECT u&\"x\" FROM t")] + public void Rejects_unicode_escaped_identifiers(string sql) + { + Assert.Contains(Compile(sql).Errors, e => e.Message.Contains("U&")); + } + [Theory] [InlineData("SELECT 'pg_terminate_backend'")] [InlineData("SELECT 1 -- pg_terminate_backend(pid)")] From 87f022ba628ece8bd02934912e2d4eae9cf6dd90 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:00:39 -0500 Subject: [PATCH 19/26] fix(cli): never repeat connection input in an error --- src/Pgcheckup/Cli/ConnectionInput.cs | 46 ++++++++++--------- .../Cli/ConnectionInputTests.cs | 33 +++++++++---- 2 files changed, 48 insertions(+), 31 deletions(-) diff --git a/src/Pgcheckup/Cli/ConnectionInput.cs b/src/Pgcheckup/Cli/ConnectionInput.cs index 739bb14..2b16ffc 100644 --- a/src/Pgcheckup/Cli/ConnectionInput.cs +++ b/src/Pgcheckup/Cli/ConnectionInput.cs @@ -4,7 +4,7 @@ namespace Pgcheckup.Cli; -// Errors name the part that is wrong and never repeat the input, which may hold a password. +// Errors say which part is wrong without repeating any of the input, which may hold a password. public sealed class ConnectionInputException(string message) : Exception(message); // Reads connections the way psql does: a postgres:// URL, a libpq key-value string, or nothing, @@ -40,7 +40,7 @@ private static readonly (string Variable, string Key)[] Environment = ["passfile"] = (b, v) => b.Passfile = v, ["sslmode"] = (b, v) => b.SslMode = SslModes.TryGetValue(v, out var mode) ? mode - : throw new ConnectionInputException($"sslmode {v} isn't one of {string.Join(", ", SslModes.Keys)}."), + : throw new ConnectionInputException($"sslmode must be one of {string.Join(", ", SslModes.Keys)}."), ["sslrootcert"] = (b, v) => b.RootCertificate = v, ["sslcert"] = (b, v) => b.SslCertificate = v, ["sslkey"] = (b, v) => b.SslKey = v, @@ -85,10 +85,11 @@ public static NpgsqlConnectionStringBuilder Parse(string? input, IReadOnlyDictio // libpq's fallback when there's no Unix socket. Npgsql has no default at all. values.TryAdd("host", "localhost"); - var unknown = values.Keys.FirstOrDefault(k => !Keywords.ContainsKey(k)); - if (unknown != null) + // An unknown key may be half of a password with a space in it, so it isn't named. + if (values.Keys.Any(k => !Keywords.ContainsKey(k))) { - throw new ConnectionInputException($"pgcheckup doesn't know the connection parameter {unknown}."); + throw new ConnectionInputException( + $"The connection has a parameter pgcheckup doesn't read. It reads {string.Join(", ", Keywords.Keys)}."); } var settings = new NpgsqlConnectionStringBuilder(); @@ -103,6 +104,22 @@ public static NpgsqlConnectionStringBuilder Parse(string? input, IReadOnlyDictio private static void ReadUrl(string url, Dictionary values) { var rest = url[(url.IndexOf("://", StringComparison.Ordinal) + 3)..]; + + // As in libpq, credentials run to the first @ that comes before any /, so a password + // may hold ? or : but a / in it must be percent-encoded. + var credentialsEnd = rest.IndexOfAny(['@', '/']); + if (credentialsEnd >= 0 && rest[credentialsEnd] == '@') + { + var userInfo = rest[..credentialsEnd]; + rest = rest[(credentialsEnd + 1)..]; + var colon = userInfo.IndexOf(':'); + values["user"] = Uri.UnescapeDataString(colon >= 0 ? userInfo[..colon] : userInfo); + if (colon >= 0) + { + values["password"] = Uri.UnescapeDataString(userInfo[(colon + 1)..]); + } + } + var query = ""; var questionMark = rest.IndexOf('?'); if (questionMark >= 0) @@ -119,19 +136,6 @@ private static void ReadUrl(string url, Dictionary values) values["dbname"] = Uri.UnescapeDataString(database); } - var at = authority.LastIndexOf('@'); - if (at >= 0) - { - var userInfo = authority[..at]; - authority = authority[(at + 1)..]; - var colon = userInfo.IndexOf(':'); - values["user"] = Uri.UnescapeDataString(colon >= 0 ? userInfo[..colon] : userInfo); - if (colon >= 0) - { - values["password"] = Uri.UnescapeDataString(userInfo[(colon + 1)..]); - } - } - if (authority.Length > 0) { var hosts = authority.Split(',').Select(SplitHostPort).ToList(); @@ -217,7 +221,7 @@ private static void ReadKeyValues(string input, Dictionary value if (i >= input.Length || input[i] != '=') { - throw new ConnectionInputException($"The connection parameter {key} has no = and value."); + throw new ConnectionInputException("Part of the connection string isn't key=value. Put values that contain spaces in single quotes."); } i++; @@ -252,7 +256,7 @@ private static void ReadKeyValues(string input, Dictionary value if (!closed) { - throw new ConnectionInputException($"The value of {key} opens a quote that never closes."); + throw new ConnectionInputException("A value in the connection string opens a quote that never closes."); } } else @@ -275,5 +279,5 @@ private static void ReadKeyValues(string input, Dictionary value private static int Number(string key, string value) => int.TryParse(value, NumberStyles.None, CultureInfo.InvariantCulture, out var number) ? number - : throw new ConnectionInputException($"The {key} {value} isn't a number."); + : throw new ConnectionInputException($"The {key} in the connection isn't a number."); } diff --git a/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs b/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs index ac44c93..ff517b0 100644 --- a/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs +++ b/tests/Pgcheckup.Tests/Cli/ConnectionInputTests.cs @@ -111,27 +111,40 @@ public void Connects_to_localhost_when_nothing_names_a_host() Assert.Equal("localhost", Parse("dbname=app").Host); } + [Fact] + public void Reads_a_password_with_a_question_mark_as_psql_does() + { + // libpq takes everything up to the first @ that comes before any / as credentials. + Assert.Equal("hun?ter2", Parse("postgres://checkup:hun?ter2@db.example.com/app").Password); + } + [Theory] - [InlineData("postgres://db.example.com/app?colour=blue", "colour")] - [InlineData("host=db.example.com colour=blue", "colour")] - [InlineData("postgres://db.example.com/app?sslmode=sometimes", "sometimes")] - [InlineData("postgres://db.example.com:abc/app", "abc")] + [InlineData("postgres://db.example.com/app?colour=blue", "parameter pgcheckup doesn't read")] + [InlineData("host=db.example.com colour=blue", "parameter pgcheckup doesn't read")] + [InlineData("postgres://db.example.com/app?sslmode=sometimes", "verify-full")] + [InlineData("postgres://db.example.com:abc/app", "port")] [InlineData("mysql://db.example.com/app", "postgres://")] [InlineData("host=db.example.com password='unterminated", "quote")] - public void Explains_what_is_wrong_with_the_input(string input, string named) + [InlineData("host=db.example.com dbname", "key=value")] + public void Explains_what_is_wrong_with_the_input(string input, string expected) { var error = Assert.Throws(() => Parse(input)); - Assert.Contains(named, error.Message); + Assert.Contains(expected, error.Message); } [Theory] - [InlineData("postgres://checkup:hunter2@db.example.com:abc/app")] - [InlineData("host=db.example.com password=hunter2 colour=blue")] - public void Never_repeats_the_password_in_an_error(string input) + [InlineData("postgres://checkup:hunter2@db.example.com:abc/app", "hunter2", "abc")] + [InlineData("postgres://checkup:hun/ter2@db.example.com/app", "hun", "ter2")] + [InlineData("host=db.example.com password=hunter2 colour=blue", "hunter2", "colour")] + [InlineData("host=db.example.com password=hun ter2", "hun", "ter2")] + [InlineData("host=db.example.com port=hunter2", "hunter2", "hunter2")] + [InlineData("host=db.example.com sslmode=hunter2", "hunter2", "hunter2")] + public void Never_repeats_any_of_the_input_in_an_error(string input, string first, string second) { var error = Assert.Throws(() => Parse(input)); - Assert.DoesNotContain("hunter2", error.Message); + Assert.DoesNotContain(first, error.Message); + Assert.DoesNotContain(second, error.Message); } } From 4dd604aaf5cc7b31f23c3eb0e0102020c5c5837e Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:00:39 -0500 Subject: [PATCH 20/26] fix(cli): exit 2 on any error a scan didn't plan for --- .../ThresholdValue.cs | 28 ++++++++++++--- src/Pgcheckup/Cli/PgcheckupCli.cs | 36 +++++++++++++++---- src/Pgcheckup/Engine/Scanner.cs | 2 +- .../Checks/ThresholdValueTests.cs | 5 +++ tests/Pgcheckup.Tests/Cli/CommandLineTests.cs | 21 +++++++++++ 5 files changed, 79 insertions(+), 13 deletions(-) diff --git a/src/Pgcheckup.Checks.Generator/ThresholdValue.cs b/src/Pgcheckup.Checks.Generator/ThresholdValue.cs index c5fc75e..35061e9 100644 --- a/src/Pgcheckup.Checks.Generator/ThresholdValue.cs +++ b/src/Pgcheckup.Checks.Generator/ThresholdValue.cs @@ -63,14 +63,19 @@ public static bool TryParse(string text, out ThresholdValue value, out string er return false; } - var number = decimal.Parse(match.Groups[1].Value, NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture); + // Checked before any unit is applied, so the multiplication below can't overflow decimal. + if (!decimal.TryParse(match.Groups[1].Value, NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var number) + || number > long.MaxValue) + { + error = $"'{text}' is too large for a threshold."; + return false; + } + var unit = match.Groups[2].Value; if (unit.Length == 0) { var kind = match.Groups[1].Value.Contains(".") ? ThresholdKind.Number : ThresholdKind.Integer; - value = new ThresholdValue(kind, number, text); - error = ""; - return true; + return InRange(new ThresholdValue(kind, number, text), out value, out error); } if (!Units.TryGetValue(unit, out var known)) @@ -79,7 +84,20 @@ public static bool TryParse(string text, out ThresholdValue value, out string er return false; } - value = new ThresholdValue(known.Kind, Math.Round(number * known.Factor, MidpointRounding.AwayFromZero), text); + return InRange(new ThresholdValue(known.Kind, Math.Round(number * known.Factor, MidpointRounding.AwayFromZero), text), out value, out error); + } + + // Thresholds bind as bigint or interval, so they must fit in 64 bits. + private static bool InRange(ThresholdValue candidate, out ThresholdValue value, out string error) + { + if (candidate.Value > long.MaxValue) + { + value = default; + error = $"'{candidate.Text}' is too large for a threshold."; + return false; + } + + value = candidate; error = ""; return true; } diff --git a/src/Pgcheckup/Cli/PgcheckupCli.cs b/src/Pgcheckup/Cli/PgcheckupCli.cs index b0ac1ad..42484f1 100644 --- a/src/Pgcheckup/Cli/PgcheckupCli.cs +++ b/src/Pgcheckup/Cli/PgcheckupCli.cs @@ -11,8 +11,18 @@ public static class PgcheckupCli public const int FindingsReachedFailOn = 1; public const int CouldNotRun = 2; + public static Task RunAsync( + string[] args, + TextWriter output, + TextWriter error, + IReadOnlyDictionary environment, + bool outputRedirected, + CancellationToken cancellationToken) => + RunAsync(args, CheckCatalog.All, output, error, environment, outputRedirected, cancellationToken); + public static async Task RunAsync( string[] args, + IReadOnlyList checks, TextWriter output, TextWriter error, IReadOnlyDictionary environment, @@ -22,7 +32,7 @@ public static async Task RunAsync( var root = new RootCommand("Checks a PostgreSQL database for the problems that cause outages. Read-only, and safe to run on production."); var list = new Command("list", "List every check."); - list.SetAction(_ => List(output)); + list.SetAction(_ => List(checks, output)); root.Subcommands.Add(list); var connection = new Argument("connection") @@ -41,7 +51,7 @@ public static async Task RunAsync( scan.Arguments.Add(connection); scan.Options.Add(failOn); scan.SetAction((result, token) => ScanAsync( - result.GetValue(connection), result.GetValue(failOn)!, output, error, environment, outputRedirected, token)); + result.GetValue(connection), result.GetValue(failOn)!, checks, output, error, environment, outputRedirected, token)); root.Subcommands.Add(scan); var parsed = root.Parse(args); @@ -58,13 +68,24 @@ public static async Task RunAsync( return CouldNotRun; } - return await parsed.InvokeAsync(new InvocationConfiguration { Output = output, Error = error }, cancellationToken); + // System.CommandLine's own handler would print a stack trace and exit 1, which means findings. + try + { + return await parsed.InvokeAsync( + new InvocationConfiguration { Output = output, Error = error, EnableDefaultExceptionHandler = false }, + cancellationToken); + } + catch (Exception problem) + { + error.WriteLine($"pgcheckup: the scan stopped: {problem.Message}"); + return CouldNotRun; + } } - private static int List(TextWriter output) + private static int List(IReadOnlyList checks, TextWriter output) { - var width = CheckCatalog.All.Max(c => c.Id.Length) + 2; - foreach (var check in CheckCatalog.All) + var width = checks.Max(c => c.Id.Length) + 2; + foreach (var check in checks) { output.WriteLine($"{check.Id.PadRight(width)}{check.Severity.ToString().ToLowerInvariant(),-10}{check.Title}"); } @@ -75,6 +96,7 @@ private static int List(TextWriter output) private static async Task ScanAsync( string? input, string failOn, + IReadOnlyList checks, TextWriter output, TextWriter error, IReadOnlyDictionary environment, @@ -109,7 +131,7 @@ private static async Task ScanAsync( ScanReport report; try { - report = await Scanner.ScanAsync(session, host, CheckCatalog.All, cancellationToken); + report = await Scanner.ScanAsync(session, host, checks, cancellationToken); } catch (Exception problem) when (problem is CheckFailedException or NpgsqlException) { diff --git a/src/Pgcheckup/Engine/Scanner.cs b/src/Pgcheckup/Engine/Scanner.cs index 3331506..2463c84 100644 --- a/src/Pgcheckup/Engine/Scanner.cs +++ b/src/Pgcheckup/Engine/Scanner.cs @@ -43,7 +43,7 @@ public static async Task ScanAsync( { results.Add(new CheckResult(check, await CheckRunner.RunAsync(session, check, cancellationToken))); } - catch (Exception error) when (error is CheckException or Npgsql.NpgsqlException) + catch (Exception error) when (error is not OperationCanceledException) { throw new CheckFailedException(check.Id, error); } diff --git a/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs b/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs index fe0c9ff..dfe2a9f 100644 --- a/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs +++ b/tests/Pgcheckup.Tests/Checks/ThresholdValueTests.cs @@ -57,6 +57,11 @@ public void Parses_a_decimal_number() [InlineData("90%")] [InlineData("h")] [InlineData("1.5.1s")] + [InlineData("99999999TB")] + [InlineData("99999999999999999999")] + [InlineData("999999999999d")] + [InlineData("99999999999999999999999999999999")] + [InlineData("99999999999999999999TB")] public void Rejects_values_postgres_would_not_accept(string text) { Assert.False(ThresholdValue.TryParse(text, out _, out var error)); diff --git a/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs b/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs index ab8c096..293b285 100644 --- a/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs +++ b/tests/Pgcheckup.Tests/Cli/CommandLineTests.cs @@ -1,3 +1,4 @@ +using Pgcheckup.Checks; using Pgcheckup.Cli; using Pgcheckup.Tests.Postgres; @@ -74,6 +75,26 @@ public async Task Reports_a_warning_and_exits_0_below_the_default_fail_on() Assert.EndsWith("0 passed · 1 warning\n", output); } + [Fact] + public async Task Exits_2_when_a_check_fails_in_a_way_nobody_planned_for() + { + // Npgsql can't read NaN into a decimal, so the runner throws something no catch expects. + var check = new CheckDefinition( + "nan-check", "NaN check", "wal", Severity.Critical, 14, [], [], [], + "SELECT 'a' AS subject, 'NaN'::numeric AS size", + new Template([new ValuePart("size", ValueFormat.Bytes)]), + new Template([new TextPart("nothing")]), + ""); + var output = new StringWriter(); + var error = new StringWriter(); + + var exitCode = await PgcheckupCli.RunAsync( + ["scan", await postgres.CheckupUrlAsync()], [check], output, error, new Dictionary(), outputRedirected: true, TestContext.Current.CancellationToken); + + Assert.Equal(2, exitCode); + Assert.StartsWith("pgcheckup: ", error.ToString()); + } + [Fact] public async Task Exits_1_when_a_finding_reaches_fail_on() { From 2c62a2dcd07e69bda1a96d8cfbbf5904895ec4bd Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:00:40 -0500 Subject: [PATCH 21/26] fix(checks): skip slots synced from the primary on a standby --- checks/replication-slot-inactive/check.sql | 3 +++ 1 file changed, 3 insertions(+) diff --git a/checks/replication-slot-inactive/check.sql b/checks/replication-slot-inactive/check.sql index a5e3170..070362f 100644 --- a/checks/replication-slot-inactive/check.sql +++ b/checks/replication-slot-inactive/check.sql @@ -14,5 +14,8 @@ CROSS JOIN LATERAL ( WHERE NOT s.active -- A lost slot has already been invalidated and holds no WAL. AND s.wal_status IS DISTINCT FROM 'lost' + -- On a standby, a slot synced from the primary (Postgres 17 and later) always looks inactive, + -- and can't be dropped there. Its consumer is on the primary, where this check covers it. + AND NOT coalesce((to_jsonb(s) ->> 'synced')::boolean, false) AND w.retained_wal >= @min_retained_wal ORDER BY w.retained_wal DESC From dfe3a495658fd9b23734340cec09c842b93e0bc3 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:00:40 -0500 Subject: [PATCH 22/26] fix(checks): reject a threshold named twice --- src/Pgcheckup.Checks.Generator/CheckCompiler.cs | 6 ++++++ tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/src/Pgcheckup.Checks.Generator/CheckCompiler.cs b/src/Pgcheckup.Checks.Generator/CheckCompiler.cs index 6971704..916d000 100644 --- a/src/Pgcheckup.Checks.Generator/CheckCompiler.cs +++ b/src/Pgcheckup.Checks.Generator/CheckCompiler.cs @@ -232,6 +232,12 @@ IReadOnlyList Template(string key) foreach (var (name, text, line) in thresholdEntry.Map) { + if (thresholdNames.Contains(name)) + { + errors.Add(new CheckError(CheckMd, line, $"The threshold {name} appears more than once.")); + continue; + } + thresholdNames.Add(name); if (!SnakeCase.IsMatch(name)) { diff --git a/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs b/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs index 8ec060b..c61d1f8 100644 --- a/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs +++ b/tests/Pgcheckup.Tests/Checks/CheckCompilerTests.cs @@ -178,6 +178,15 @@ public void Reports_a_bad_threshold_on_its_line() Assert.Equal(10, error.Line); } + [Fact] + public void Reports_a_threshold_named_twice() + { + var error = SingleError(Compile(Markdown(ValidFrontmatter.Replace(" min_age: 1h", " min_age: 1h\n min_age: 2h")))); + + Assert.Equal(11, error.Line); + Assert.Contains("min_age", error.Message); + } + [Fact] public void Reports_template_errors_on_the_template_line() { From c894e784b03a90eca17dc72d5a2567a015ae49a5 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 02:32:47 -0500 Subject: [PATCH 23/26] docs(roadmap): tick CI for M0 --- ROADMAP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ROADMAP.md b/ROADMAP.md index 00ff44c..4f70603 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -40,7 +40,7 @@ pgcheckup is a read-only CLI (.NET 10, NativeAOT) that checks a PostgreSQL datab - [x] Fixture runner on Testcontainers: for every check, `fires.sql` must produce a finding and `healthy.sql` must not. - [x] Read-only guard: the fixture runner runs each check as a role that has only `pg_monitor`, inside a `READ ONLY` transaction. Positive control: a test check that writes must fail. - [x] `pgcheckup scan` with terminal output and exit codes 0, 1 and 2. -- [ ] CI on every PR: build, fixture tests on Postgres 14 to 18, and a NativeAOT publish that fails on any IL2xxx or IL3xxx warning. +- [x] CI on every PR: build, fixture tests on Postgres 14 to 18, and a NativeAOT publish that fails on any IL2xxx or IL3xxx warning. - [x] Brand: the logo in `docs/brand/`, with `-dark` variants. Then replace the README heading with a `` lockup. - [x] Add a recording or screenshot of a scan to the README. From 89139561636ee9bee4dc57013960989f487d70da Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 08:48:29 -0500 Subject: [PATCH 24/26] docs(code): add XML docs to the public API --- .../CheckCompiler.cs | 64 ++++++++++++++++++- .../CheckGenerator.cs | 12 ++++ src/Pgcheckup.Checks.Generator/CheckSql.cs | 16 +++++ src/Pgcheckup.Checks.Generator/Frontmatter.cs | 40 ++++++++++-- .../SqlTokenizer.cs | 43 ++++++++++++- .../TemplateParser.cs | 24 ++++++- .../ThresholdValue.cs | 35 +++++++++- src/Pgcheckup/Checks/CheckDefinition.cs | 34 +++++++++- src/Pgcheckup/Checks/Template.cs | 25 ++++++++ src/Pgcheckup/Checks/ValueText.cs | 16 ++++- src/Pgcheckup/Cli/ConnectionInput.cs | 24 +++++-- src/Pgcheckup/Cli/PgcheckupCli.cs | 26 ++++++++ src/Pgcheckup/Cli/TerminalReport.cs | 16 +++++ src/Pgcheckup/Engine/CheckRunner.cs | 19 +++++- src/Pgcheckup/Engine/ReadOnlySession.cs | 30 ++++++++- src/Pgcheckup/Engine/Scanner.cs | 25 +++++++- 16 files changed, 425 insertions(+), 24 deletions(-) diff --git a/src/Pgcheckup.Checks.Generator/CheckCompiler.cs b/src/Pgcheckup.Checks.Generator/CheckCompiler.cs index 916d000..0d96d88 100644 --- a/src/Pgcheckup.Checks.Generator/CheckCompiler.cs +++ b/src/Pgcheckup.Checks.Generator/CheckCompiler.cs @@ -5,82 +5,137 @@ namespace Pgcheckup.Checks.Generator; +/// The files of one checks/<id>/ folder, as the compiler needs them. +/// The folder name, which must match the id in check.md. +/// The contents of check.md, or when the folder has none. +/// The contents of check.sql, or when the folder has none. +/// Every file in the folder, relative to it, with forward slashes. public sealed class CheckFiles(string id, string? checkMd, string? checkSql, IReadOnlyCollection otherFiles) { + /// The folder name, which must match the id in check.md. public string Id { get; } = id; + /// The contents of check.md, or when the folder has none. public string? CheckMd { get; } = checkMd; + /// The contents of check.sql, or when the folder has none. public string? CheckSql { get; } = checkSql; - // Paths relative to the check's folder, with forward slashes. + /// Every file in the folder, relative to it, with forward slashes, such as fixtures/fires.sql. public IReadOnlyCollection OtherFiles { get; } = otherFiles; } +/// A problem with a check, located in one of its files. The generator reports it as build error PGC001. +/// The file, relative to the check's folder, such as check.md. +/// The 1-based line in that file. +/// What is wrong, in a sentence a check author can act on. public sealed class CheckError(string file, int line, string message) { + /// The file, relative to the check's folder, such as check.md. public string File { get; } = file; + /// The 1-based line in that file. public int Line { get; } = line; + /// What is wrong, in a sentence a check author can act on. public string Message { get; } = message; + /// public override string ToString() => $"{File}({Line}): {Message}"; } +/// A threshold that check.sql reads. +/// The name that check.sql reads as @name. +/// Its default value. public sealed class CompiledThreshold(string name, ThresholdValue value) { + /// The name that check.sql reads as @name. public string Name { get; } = name; + /// Its default value. public ThresholdValue Value { get; } = value; } +/// A check that passed every build-time rule, ready to be emitted into the binary. public sealed class CompiledCheck { + /// The check's stable kebab-case id, equal to its folder name. public string Id { get; set; } = ""; + /// A short title for pgcheckup list. public string Title { get; set; } = ""; + /// One of . public string Category { get; set; } = ""; + /// The default severity of a finding, one of . public string Severity { get; set; } = ""; + /// The oldest Postgres major version the check runs on. public int MinVersion { get; set; } + /// The predefined roles the check needs, from . May be empty. public IReadOnlyList Privileges { get; set; } = []; + /// The providers where the check is skipped, from . public IReadOnlyList SkipOn { get; set; } = []; - // In the order of their $n parameters. + /// The thresholds in the order of their $n parameters, so index 0 binds to $1. public IReadOnlyList Thresholds { get; set; } = []; + /// check.sql with thresholds rewritten to $n parameters. public string Sql { get; set; } = ""; + /// The parsed message template. public IReadOnlyList Message { get; set; } = []; + /// The parsed fix template. public IReadOnlyList Fix { get; set; } = []; + /// The Markdown body of check.md, which pgcheckup explain prints. public string Note { get; set; } = ""; } +/// The outcome of . +/// The compiled check, or when there are errors. +/// Every problem found. Empty when the check compiled. public sealed class CheckCompilation(CompiledCheck? check, IReadOnlyList errors) { + /// The compiled check, or when there are errors. public CompiledCheck? Check { get; } = check; + /// Every problem found. Empty when the check compiled. public IReadOnlyList Errors { get; } = errors; } +/// Applies every build-time rule to one check folder. public static class CheckCompiler { + /// The note and frontmatter file. public const string CheckMd = "check.md"; + + /// The query file. public const string CheckSqlFile = "check.sql"; + + /// The fixture the check must report on. public const string FiresFixture = "fixtures/fires.sql"; + + /// The fixture the check must stay quiet on. public const string HealthyFixture = "fixtures/healthy.sql"; + /// The allowed values of category. public static readonly string[] Categories = ["ids", "cleanup", "wal", "capacity"]; + + /// The allowed values of severity, from most to least severe. public static readonly string[] Severities = ["critical", "warning", "info"]; + + /// The predefined roles a check may list under privileges. All are part of pg_monitor. public static readonly string[] Privileges = ["pg_monitor", "pg_read_all_settings", "pg_read_all_stats", "pg_stat_scan_tables"]; + + /// The managed providers a check may list under skip_on. public static readonly string[] Providers = ["rds", "aurora", "cloudsql", "azure", "supabase", "neon"]; + + /// The ## sections every check.md body must have. public static readonly string[] Sections = ["What breaks", "Fix", "Seen in"]; private static readonly string[] RequiredKeys = ["id", "title", "category", "severity", "min_version", "privileges", "message", "fix"]; @@ -89,6 +144,11 @@ public static class CheckCompiler private static readonly Regex KebabCase = new("^[a-z0-9]+(-[a-z0-9]+)*$", RegexOptions.CultureInvariant); private static readonly Regex SnakeCase = new("^[a-z][a-z0-9_]*$", RegexOptions.CultureInvariant); + /// + /// Compiles one check folder: its frontmatter, templates, body sections, SQL and fixtures. + /// + /// The folder's files. + /// The compiled check, or every error found. Errors don't stop at the first one. public static CheckCompilation Compile(CheckFiles files) { var errors = new List(); diff --git a/src/Pgcheckup.Checks.Generator/CheckGenerator.cs b/src/Pgcheckup.Checks.Generator/CheckGenerator.cs index fc3d0e4..2233637 100644 --- a/src/Pgcheckup.Checks.Generator/CheckGenerator.cs +++ b/src/Pgcheckup.Checks.Generator/CheckGenerator.cs @@ -10,6 +10,15 @@ namespace Pgcheckup.Checks.Generator; +/// +/// Compiles every folder under checks/ into the CheckCatalog class, so the binary +/// carries its checks and parses nothing at runtime. +/// +/// +/// The checks folder comes from the PgcheckupChecksDir MSBuild property, and its files +/// from AdditionalFiles. Every becomes build error PGC001 on the +/// file and line it names. +/// [Generator(LanguageNames.CSharp)] public sealed class CheckGenerator : IIncrementalGenerator { @@ -21,6 +30,7 @@ public sealed class CheckGenerator : IIncrementalGenerator private static readonly DiagnosticDescriptor NoChecksDir = new( "PGC002", "Checks folder not set", "{0}", "pgcheckup", DiagnosticSeverity.Error, isEnabledByDefault: true); + /// public void Initialize(IncrementalGeneratorInitializationContext context) { var root = context.AnalyzerConfigOptionsProvider.Select((options, _) => @@ -83,8 +93,10 @@ private static string Source(IReadOnlyList checks) code.AppendLine("#nullable enable"); code.AppendLine("namespace Pgcheckup.Checks;"); code.AppendLine(); + code.AppendLine("/// The checks compiled from the checks folder at build time."); code.AppendLine("internal static partial class CheckCatalog"); code.AppendLine("{"); + code.AppendLine(" /// Every check, ordered by id."); code.AppendLine($" public static global::System.Collections.Generic.IReadOnlyList<{ns}CheckDefinition> All {{ get; }} = new {ns}CheckDefinition[]"); code.AppendLine(" {"); foreach (var check in checks) diff --git a/src/Pgcheckup.Checks.Generator/CheckSql.cs b/src/Pgcheckup.Checks.Generator/CheckSql.cs index 98da3ed..602122b 100644 --- a/src/Pgcheckup.Checks.Generator/CheckSql.cs +++ b/src/Pgcheckup.Checks.Generator/CheckSql.cs @@ -5,8 +5,13 @@ namespace Pgcheckup.Checks.Generator; +/// The outcome of compiling a check.sql with . public sealed class SqlResult { + /// Creates a result. + /// The SQL with @name rewritten to $n and a trailing semicolon removed. + /// The threshold names, where the name at index 0 binds to $1. + /// Everything wrong with the SQL. Empty when it compiled. public SqlResult(string sql, IReadOnlyList parameters, IReadOnlyList errors) { Sql = sql; @@ -14,13 +19,17 @@ public SqlResult(string sql, IReadOnlyList parameters, IReadOnlyListThe SQL with @name rewritten to $n and a trailing semicolon removed. public string Sql { get; } + /// The threshold names in order of first use. The name at index 0 binds to $1. public IReadOnlyList Parameters { get; } + /// Everything wrong with the SQL. Empty when it compiled. public IReadOnlyList Errors { get; } } +/// Checks a check.sql against the read-only rules and prepares it to run. public static class CheckSql { // A READ ONLY transaction blocks DDL, DML, nextval and row locks, but not these. They signal @@ -46,6 +55,13 @@ public static class CheckSql "query_to_xml", "table_to_xml", "cursor_to_xml", "schema_to_xml", "database_to_xml", ]; + /// + /// Compiles a check.sql. It must be one statement that starts with SELECT or WITH, + /// read every declared threshold and nothing else, and call no function on the deny list. + /// + /// The contents of check.sql. + /// The threshold names declared in check.md. + /// The rewritten SQL, its parameter order and every error found. public static SqlResult Compile(string sql, IReadOnlyCollection thresholds) { var errors = new List(); diff --git a/src/Pgcheckup.Checks.Generator/Frontmatter.cs b/src/Pgcheckup.Checks.Generator/Frontmatter.cs index ffa5547..1f14165 100644 --- a/src/Pgcheckup.Checks.Generator/Frontmatter.cs +++ b/src/Pgcheckup.Checks.Generator/Frontmatter.cs @@ -5,48 +5,80 @@ namespace Pgcheckup.Checks.Generator; +/// The shape of a frontmatter value. public enum EntryKind { + /// One value: plain, quoted, or a | or > block. Scalar, + + /// A [a, b] list on one line. List, + + /// Indented name: value lines under an empty key. Map, } +/// One top-level key of a check's frontmatter and its value. +/// The key. +/// The 1-based line the key is on. +/// The shape of its value. public sealed class FrontmatterEntry(string key, int line, EntryKind kind) { + /// The key. public string Key { get; } = key; + /// The 1-based line the key is on. public int Line { get; } = line; - // For block scalars, the line where the content starts. + /// The line where the value starts: the key's line, or the next one for a block scalar. public int ValueLine { get; set; } = line; + /// The shape of the value, which says which of , and is set. public EntryKind Kind { get; } = kind; + /// The value of a entry, unquoted, with a block scalar joined and trimmed. public string Scalar { get; set; } = ""; + /// The items of a entry, unquoted. public List Items { get; } = []; + /// The entries of a entry, in order and with their lines. Duplicates are kept. public List<(string Key, string Value, int Line)> Map { get; } = []; } +/// A check.md split into its frontmatter entries and its Markdown body. public sealed class FrontmatterDocument { + /// The top-level entries in order. A duplicate key is kept and reported in . public List Entries { get; } = []; + /// The Markdown after the closing --- line, trimmed. public string Body { get; set; } = ""; + /// Everything that isn't valid frontmatter. Empty when the frontmatter parsed. public List Errors { get; } = []; } -// A strict subset of YAML: top-level `key: value`, `[a, b]` lists, one level of nested -// `name: value` maps, quoted scalars, and `|` or `>` block scalars. Anything else is an error -// with a line number, rather than something a full YAML parser would read differently. +/// +/// Reads check.md frontmatter: a strict subset of YAML with top-level key: value, +/// [a, b] lists, one level of nested name: value maps, quoted scalars, and +/// | or > block scalars. +/// +/// +/// Anything outside the subset is an error with a line number, rather than something a full +/// YAML parser would read differently. +/// public static class Frontmatter { private static readonly Regex TopLevel = new(@"^([A-Za-z_][A-Za-z0-9_]*):(.*)$", RegexOptions.CultureInvariant); private static readonly Regex Nested = new(@"^\s+([^:\s]+):(.*)$", RegexOptions.CultureInvariant); + /// Parses a check.md. + /// The file's contents. CRLF line endings are accepted. + /// + /// The entries, body and errors. When the file doesn't start with a frontmatter block, + /// the only error is on line 1 and there are no entries. + /// public static FrontmatterDocument Parse(string markdown) { var document = new FrontmatterDocument(); diff --git a/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs b/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs index 9b9f52b..ecd3539 100644 --- a/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs +++ b/src/Pgcheckup.Checks.Generator/SqlTokenizer.cs @@ -2,33 +2,58 @@ namespace Pgcheckup.Checks.Generator; +/// A problem found in one file of a check, with the line it is on. public sealed class SourceError { + /// Creates an error for a line of the file being read. + /// The 1-based line number. + /// What is wrong, in a sentence a check author can act on. public SourceError(int line, string message) { Line = line; Message = message; } + /// The 1-based line number the error is on. public int Line { get; } + /// What is wrong, in a sentence a check author can act on. public string Message { get; } + /// public override string ToString() => $"line {Line}: {Message}"; } +/// The kinds of token tells apart. public enum TokenKind { + /// A keyword or unquoted identifier, such as SELECT or pg_stat_activity. Word, + + /// A double-quoted identifier. holds the name without quotes. QuotedIdentifier, + + /// A threshold reference such as @min_age. holds the name without @. Parameter, + + /// A positional parameter such as $1, which check.sql must not use. PositionalParameter, + + /// A ; outside any literal or comment. Semicolon, + + /// Any other single character: operators, punctuation and digits. Other, } +/// One token of a SQL text, with its position in that text. public readonly struct Token { + /// Creates a token. + /// What kind of token it is. + /// The 0-based offset of its first character in the SQL text. + /// How many characters of the SQL text it covers. + /// Its text, unquoted for identifiers and without @ for parameters. public Token(TokenKind kind, int start, int length, string text) { Kind = kind; @@ -37,19 +62,32 @@ public Token(TokenKind kind, int start, int length, string text) Text = text; } + /// What kind of token it is. public TokenKind Kind { get; } + /// The 0-based offset of its first character in the SQL text. public int Start { get; } + /// How many characters of the SQL text it covers, including quotes or @. public int Length { get; } + /// Its text, unquoted for identifiers and without @ for parameters. public string Text { get; } } -// Just enough of Postgres's lexer to tell code from comments, string literals and quoted -// identifiers, so that checks on statements and function names can't be fooled by either. +/// +/// Just enough of Postgres's lexer to tell code from comments, string literals and quoted +/// identifiers, so that checks on statements and function names can't be fooled by either. +/// public static class SqlTokenizer { + /// Splits SQL into tokens, skipping whitespace, comments and string literals. + /// The SQL text, such as a check.sql or a fixture. + /// Receives a for each comment, string or identifier that is never closed. + /// + /// The tokens in order. String literals (standard, E'' and dollar-quoted) and comments + /// produce no tokens. + /// public static List Tokenize(string sql, List errors) { var tokens = new List(); @@ -233,6 +271,7 @@ private static bool TryReadDollarTag(string sql, int i, out string tag) private static bool IsIdentifierPart(char c) => IsIdentifierStart(c) || char.IsDigit(c) || c == '$'; + /// The 1-based line that a 0-based offset falls on. internal static int LineOf(string text, int position) { var line = 1; diff --git a/src/Pgcheckup.Checks.Generator/TemplateParser.cs b/src/Pgcheckup.Checks.Generator/TemplateParser.cs index 02cb624..13fa5be 100644 --- a/src/Pgcheckup.Checks.Generator/TemplateParser.cs +++ b/src/Pgcheckup.Checks.Generator/TemplateParser.cs @@ -5,35 +5,55 @@ namespace Pgcheckup.Checks.Generator; +/// One piece of a parsed message or fix template. public abstract class TemplatePart { } +/// Literal text, copied as written. +/// The text, with doubled braces and brackets already reduced to one. public sealed class TextPart(string text) : TemplatePart { + /// The text, with doubled braces and brackets already reduced to one. public string Text { get; } = text; } +/// A {name} or {name:format} placeholder for a column of the check's query. +/// The column name. +/// One of , or to format by the value's type. public sealed class ValuePart(string name, string? format) : TemplatePart { + /// The column name. public string Name { get; } = name; + /// One of , or to format by the value's type. public string? Format { get; } = format; } +/// A [ … ] section, left out of the message when any value inside it is NULL. +/// Its text and values. Sections don't nest. public sealed class SectionPart(IReadOnlyList parts) : TemplatePart { + /// Its text and values. Sections don't nest. public IReadOnlyList Parts { get; } = parts; } -// Message and fix templates: {name} or {name:format} inserts a value, [ … ] is left out when a -// value inside it is NULL, and doubled braces or brackets stand for themselves. +/// +/// Parses message and fix templates: {name} or {name:format} inserts a value, +/// [ … ] is left out when a value inside it is NULL, and doubled braces or brackets +/// stand for themselves. +/// public static class TemplateParser { + /// The formats a placeholder may name: bytes prints "48 GB" and count prints "1.61 billion". public static readonly string[] Formats = ["bytes", "count"]; private static readonly Regex Placeholder = new(@"^([a-z_][a-z0-9_]*)(?::([a-z]+))?$", RegexOptions.CultureInvariant); + /// Parses a template from check.md. + /// The template text. + /// Receives every problem found. Empty when the template is valid. + /// The parts in order. Only trust them when is empty. public static IReadOnlyList Parse(string template, out List errors) { errors = []; diff --git a/src/Pgcheckup.Checks.Generator/ThresholdValue.cs b/src/Pgcheckup.Checks.Generator/ThresholdValue.cs index 35061e9..c10a830 100644 --- a/src/Pgcheckup.Checks.Generator/ThresholdValue.cs +++ b/src/Pgcheckup.Checks.Generator/ThresholdValue.cs @@ -5,16 +5,29 @@ namespace Pgcheckup.Checks.Generator; +/// What a threshold measures, which decides the parameter type it binds as. public enum ThresholdKind { + /// A size in bytes, written with B, kB, MB, GB or TB. Binds as bigint. Bytes, + + /// A length of time, written with us, ms, s, min, h or d. Binds as interval. Duration, + + /// A whole number without a unit. Binds as bigint. Integer, + + /// A number with a decimal point and no unit. Binds as numeric. Number, } -// Bytes and durations use Postgres's own units (as in postgresql.conf), so a threshold reads -// the same as the setting it is usually compared with. Durations are held in microseconds. +/// +/// A threshold from a check's frontmatter, such as 1GB or 30min, parsed at build time. +/// +/// +/// Bytes and durations use Postgres's own units (as in postgresql.conf), so a threshold reads the +/// same as the setting it is usually compared with. Units are case-sensitive, and 1 kB is 1024 bytes. +/// public readonly struct ThresholdValue { private static readonly Regex Pattern = new(@"^(\d+(?:\.\d+)?)\s*([A-Za-z]*)$", RegexOptions.CultureInvariant); @@ -34,6 +47,10 @@ public readonly struct ThresholdValue ["d"] = (ThresholdKind.Duration, 86_400_000_000m), }; + /// Creates a threshold that has already been parsed. + /// What the threshold measures. + /// The value in bytes, microseconds, or as written for plain numbers. + /// The threshold as written in check.md, for messages. public ThresholdValue(ThresholdKind kind, decimal value, string text) { Kind = kind; @@ -41,12 +58,26 @@ public ThresholdValue(ThresholdKind kind, decimal value, string text) Text = text; } + /// What the threshold measures. public ThresholdKind Kind { get; } + /// + /// The value in the kind's base unit: bytes for , microseconds + /// for . Always fits in a 64-bit integer. + /// public decimal Value { get; } + /// The threshold as written in check.md. public string Text { get; } + /// Parses a threshold such as 1GB, 30min, 1500000000 or 0.9. + /// The value from check.md or a fixture's -- threshold line. + /// The parsed threshold, or default when parsing fails. + /// Why the text isn't a threshold, or an empty string on success. + /// + /// for a non-negative number with an optional known unit that fits in + /// 64 bits once converted to the base unit. + /// public static bool TryParse(string text, out ThresholdValue value, out string error) { value = default; diff --git a/src/Pgcheckup/Checks/CheckDefinition.cs b/src/Pgcheckup/Checks/CheckDefinition.cs index 5739efe..33ec3f3 100644 --- a/src/Pgcheckup/Checks/CheckDefinition.cs +++ b/src/Pgcheckup/Checks/CheckDefinition.cs @@ -1,24 +1,54 @@ namespace Pgcheckup.Checks; -// Ordered so that a higher value is more severe. +/// How urgent a finding is. A higher value is more severe, so severities compare directly. public enum Severity { + /// Housekeeping. Info, + + /// Heading toward an outage, or a safety net is gone. Warning, + + /// Can take the database down or lose data soon. Critical, } +/// What a threshold measures, which decides the parameter type it binds as. public enum ThresholdKind { + /// A size in bytes. Binds as bigint. Bytes, + + /// A length of time, held in microseconds. Binds as interval. Duration, + + /// A whole number. Binds as bigint. Integer, + + /// A decimal number. Binds as numeric. Number, } -// Durations are held in microseconds, as the generator parsed them. +/// A threshold that a check's query reads as a parameter. +/// The name, as check.sql reads it with @name. +/// What it measures. +/// The value in bytes, microseconds, or as written for plain numbers. +/// The value as written in check.md, such as 1GB. public sealed record Threshold(string Name, ThresholdKind Kind, decimal Value, string Text); +/// A check, compiled from its checks/<id>/ folder into the binary at build time. +/// The stable kebab-case id. Baselines and ignore lists depend on it. +/// A short title for pgcheckup list. +/// The group the check belongs to: ids, cleanup, wal or capacity. +/// The severity of a finding unless its row says otherwise. +/// The oldest Postgres major version the check runs on. +/// The predefined roles the check needs. Empty when any role can run it. +/// The managed providers where the check is skipped. +/// The thresholds in parameter order, so index 0 binds to $1. +/// One read-only statement that returns a row per finding. +/// What is wrong, rendered from each row. +/// What to do about it, rendered from each row. pgcheckup prints it and never runs it. +/// The Markdown body of check.md. public sealed record CheckDefinition( string Id, string Title, diff --git a/src/Pgcheckup/Checks/Template.cs b/src/Pgcheckup/Checks/Template.cs index 5579612..163133a 100644 --- a/src/Pgcheckup/Checks/Template.cs +++ b/src/Pgcheckup/Checks/Template.cs @@ -2,27 +2,52 @@ namespace Pgcheckup.Checks; +/// How a template prints a value. public enum ValueFormat { + /// By the value's type: intervals as "3 days", timestamps in UTC, everything else plainly. Default, + + /// A size in bytes, such as "48 GB". Written {name:bytes}. Bytes, + + /// A count, with words past a million, such as "1.61 billion". Written {name:count}. Count, } +/// One piece of a compiled message or fix template. public abstract record TemplatePart; +/// Literal text. +/// The text, printed as it is. public sealed record TextPart(string Text) : TemplatePart; +/// A value from a column of the check's query. +/// The column name. +/// How to print the value. public sealed record ValuePart(string Name, ValueFormat Format) : TemplatePart; +/// A section that is left out when any value inside it is NULL. +/// Its text and values. Sections don't nest. public sealed record SectionPart(IReadOnlyList Parts) : TemplatePart; +/// A template and a query row that don't fit: a missing column, or NULL outside a section. +/// Which value is at fault. public sealed class TemplateException(string message) : Exception(message); +/// A message or fix template, compiled from check.md at build time. +/// The template's parts in order. public sealed class Template(IReadOnlyList parts) { + /// The template's parts in order. public IReadOnlyList Parts { get; } = parts; + /// Renders the template with one row of the check's query. + /// The row, by column name. SQL NULL is . + /// The text, with each section left out when a value in it is NULL. + /// + /// The row has no column for a value, or a value outside a section is NULL. + /// public string Render(IReadOnlyDictionary values) { var text = new StringBuilder(); diff --git a/src/Pgcheckup/Checks/ValueText.cs b/src/Pgcheckup/Checks/ValueText.cs index a914a12..5d9ae33 100644 --- a/src/Pgcheckup/Checks/ValueText.cs +++ b/src/Pgcheckup/Checks/ValueText.cs @@ -2,11 +2,17 @@ namespace Pgcheckup.Checks; +/// Prints values the same way in every check, so numbers and durations read alike. public static class ValueText { private static readonly string[] ByteUnits = ["bytes", "kB", "MB", "GB", "TB", "PB"]; private static readonly string[] CountUnits = ["million", "billion", "trillion"]; + /// Prints a value from a query row. + /// A non-null value as Npgsql read it. + /// How to print it. + /// The text, without culture-specific formatting. + /// needs a number and isn't one. public static string Format(object value, ValueFormat format) => format switch { ValueFormat.Bytes => Bytes(ToDecimal(value)), @@ -21,7 +27,9 @@ public static class ValueText }, }; - // Postgres's size units (1024-based, as in pg_size_pretty), with three significant digits. + /// Prints a size with Postgres's units (1024-based, as in pg_size_pretty) and three significant digits. + /// The size in bytes. + /// Such as "512 bytes", "1.5 GB" or "48 GB". public static string Bytes(decimal bytes) { if (bytes < 1024) @@ -40,6 +48,9 @@ public static string Bytes(decimal bytes) return $"{Significant(value)} {ByteUnits[unit]}"; } + /// Prints a count with thousands separators, or in words from a million on. + /// The count. + /// Such as "48,213", "48 million" or "1.61 billion". public static string Count(decimal count) { if (count < 1_000_000) @@ -58,6 +69,9 @@ public static string Count(decimal count) return $"{Significant(value)} {CountUnits[unit]}"; } + /// Prints a duration in its largest whole unit, rounded down. + /// The duration. A negative one prints as zero. + /// Such as "3 days", "1 hour" or "45 seconds". public static string Duration(TimeSpan span) { if (span < TimeSpan.Zero) diff --git a/src/Pgcheckup/Cli/ConnectionInput.cs b/src/Pgcheckup/Cli/ConnectionInput.cs index 2b16ffc..e39dc15 100644 --- a/src/Pgcheckup/Cli/ConnectionInput.cs +++ b/src/Pgcheckup/Cli/ConnectionInput.cs @@ -4,12 +4,17 @@ namespace Pgcheckup.Cli; -// Errors say which part is wrong without repeating any of the input, which may hold a password. +/// A connection string or URL that can't be read. +/// +/// Which part is wrong. It never repeats any of the input, which may hold a password. +/// public sealed class ConnectionInputException(string message) : Exception(message); -// Reads connections the way psql does: a postgres:// URL, a libpq key-value string, or nothing, -// with PGHOST, PGPORT, PGDATABASE and PGSSLMODE filling in what the input leaves out. Npgsql -// reads PGUSER, PGPASSWORD, PGPASSFILE and ~/.pgpass itself. +/// +/// Reads connections the way psql does: a postgres:// URL, a libpq key-value string, or +/// nothing, with PGHOST, PGPORT, PGDATABASE and PGSSLMODE filling in what the input leaves out. +/// +/// Npgsql reads PGUSER, PGPASSWORD, PGPASSFILE and ~/.pgpass itself. public static class ConnectionInput { private static readonly (string Variable, string Key)[] Environment = @@ -54,6 +59,17 @@ private static readonly (string Variable, string Key)[] Environment = ["fallback_application_name"] = (_, _) => { }, }; + /// Turns what the user passed into Npgsql connection settings. + /// + /// A postgres:// or postgresql:// URL, a libpq key-value string, or + /// or blank to use the environment alone. + /// + /// The process's environment variables. + /// The settings, with the host defaulting to localhost. + /// + /// The input isn't a URL or key-value string, or has a parameter pgcheckup doesn't read, a + /// port or timeout that isn't a number, or an unknown sslmode. + /// public static NpgsqlConnectionStringBuilder Parse(string? input, IReadOnlyDictionary environment) { var values = new Dictionary(StringComparer.Ordinal); diff --git a/src/Pgcheckup/Cli/PgcheckupCli.cs b/src/Pgcheckup/Cli/PgcheckupCli.cs index 42484f1..07ac8c3 100644 --- a/src/Pgcheckup/Cli/PgcheckupCli.cs +++ b/src/Pgcheckup/Cli/PgcheckupCli.cs @@ -5,12 +5,26 @@ namespace Pgcheckup.Cli; +/// The pgcheckup command line: its commands, options and exit codes. public static class PgcheckupCli { + /// Exit code 0: the scan ran and no finding reached --fail-on. public const int Passed = 0; + + /// Exit code 1: at least one finding reached --fail-on. public const int FindingsReachedFailOn = 1; + + /// Exit code 2: the scan couldn't run, whatever the reason. public const int CouldNotRun = 2; + /// Runs pgcheckup with every check compiled into the binary. + /// The command-line arguments. + /// Where the report and help go. + /// Where errors go. + /// The process's environment variables, for PG* settings and NO_COLOR. + /// Whether is a file or pipe, which turns color off. + /// Cancels the scan. + /// 0, 1 or 2. See , and . public static Task RunAsync( string[] args, TextWriter output, @@ -20,6 +34,18 @@ public static Task RunAsync( CancellationToken cancellationToken) => RunAsync(args, CheckCatalog.All, output, error, environment, outputRedirected, cancellationToken); + /// Runs pgcheckup with the given checks. + /// The command-line arguments. + /// The checks that list shows and scan runs. + /// Where the report and help go. + /// Where errors go. + /// The process's environment variables, for PG* settings and NO_COLOR. + /// Whether is a file or pipe, which turns color off. + /// Cancels the scan. + /// + /// 0, 1 or 2. Anything that stops the scan, including bad arguments and unexpected errors, + /// returns 2, never 1. + /// public static async Task RunAsync( string[] args, IReadOnlyList checks, diff --git a/src/Pgcheckup/Cli/TerminalReport.cs b/src/Pgcheckup/Cli/TerminalReport.cs index 801b4bb..fb6ad01 100644 --- a/src/Pgcheckup/Cli/TerminalReport.cs +++ b/src/Pgcheckup/Cli/TerminalReport.cs @@ -3,16 +3,32 @@ namespace Pgcheckup.Cli; +/// The report pgcheckup scan prints to a terminal. public static class TerminalReport { private const int Indent = 10; private const string FixLabel = "Fix: "; + /// Whether to color the severity words. + /// Whether the report goes to a file or pipe. + /// The process's environment variables. + /// + /// when output is redirected, NO_COLOR is set to anything but an + /// empty string, or TERM is dumb. + /// public static bool UseColor(bool outputRedirected, IReadOnlyDictionary environment) => !outputRedirected && !(environment.TryGetValue("NO_COLOR", out var noColor) && !string.IsNullOrEmpty(noColor)) && !(environment.TryGetValue("TERM", out var term) && term == "dumb"); + /// + /// Writes the header, then each finding with its fix, then a summary that counts each check + /// once at its worst severity. + /// + /// Where to write. + /// What the scan found. + /// Whether to color the severity words. The words are always there. + /// Findings are ordered by severity, most severe first, then by check id. public static void Write(TextWriter output, ScanReport report, bool color) { var server = report.Server; diff --git a/src/Pgcheckup/Engine/CheckRunner.cs b/src/Pgcheckup/Engine/CheckRunner.cs index 50fda5c..13f9b1a 100644 --- a/src/Pgcheckup/Engine/CheckRunner.cs +++ b/src/Pgcheckup/Engine/CheckRunner.cs @@ -2,6 +2,13 @@ namespace Pgcheckup.Engine; +/// One problem a check found, ready to print. +/// The id of the check that found it. +/// The object it is about, such as a slot or table name. +/// How urgent it is. +/// What is wrong and why it matters. +/// What to do about it. pgcheckup prints it and never runs it. +/// The row the check returned, by column name, for machine-readable output. public sealed record Finding( string CheckId, string Subject, @@ -10,11 +17,21 @@ public sealed record Finding( string Fix, IReadOnlyDictionary Values); -// A check whose query or template doesn't hold up its side of the contract. +/// A check whose query or template doesn't hold up its side of the contract. +/// What the check did wrong, naming the check. +/// The error that exposed it, if any. public sealed class CheckException(string message, Exception? inner = null) : Exception(message, inner); +/// Runs one check and turns its rows into findings. public static class CheckRunner { + /// Runs a check's query with its thresholds and renders a finding per row. + /// The guarded session to query through. + /// The check to run. + /// Cancels the query. + /// A finding per row, in the query's order. Empty when the check passes. + /// A row has no subject, an unknown severity, or doesn't fit a template. + /// The query failed, timed out or couldn't get a lock. public static async Task> RunAsync(ReadOnlySession session, CheckDefinition check, CancellationToken cancellationToken) { var parameters = check.Thresholds.Select(ParameterValue).ToList(); diff --git a/src/Pgcheckup/Engine/ReadOnlySession.cs b/src/Pgcheckup/Engine/ReadOnlySession.cs index 29dc775..5c64bc0 100644 --- a/src/Pgcheckup/Engine/ReadOnlySession.cs +++ b/src/Pgcheckup/Engine/ReadOnlySession.cs @@ -2,9 +2,14 @@ namespace Pgcheckup.Engine; -// The only way pgcheckup talks to Postgres. Every query runs in its own READ ONLY transaction -// with transaction-local timeouts, then rolls back. Nothing is set for the session: behind a -// transaction pooler, a session setting would reach the application's next transaction. +/// +/// The only way pgcheckup talks to Postgres. Every query runs in its own READ ONLY +/// transaction with transaction-local timeouts and search path, then rolls back. +/// +/// +/// Nothing is set for the session: behind a transaction pooler, a session setting would reach +/// the application's next transaction. +/// public sealed class ReadOnlySession : IAsyncDisposable { private static readonly string[] Guards = @@ -29,6 +34,11 @@ private ReadOnlySession(NpgsqlDataSource dataSource, NpgsqlConnection connection this.connection = connection; } + /// Connects as application_name = pgcheckup, without loading types or pooling. + /// Where and how to connect. The caller's builder isn't changed. + /// Cancels connecting. + /// An open session. Dispose it to close the connection. + /// The server can't be reached, or refuses the login. public static async Task OpenAsync(NpgsqlConnectionStringBuilder settings, CancellationToken cancellationToken) { var builder = new NpgsqlSlimDataSourceBuilder(settings.ConnectionString); @@ -52,6 +62,18 @@ public static async Task OpenAsync(NpgsqlConnectionStringBuilde } } + /// Runs one statement in its own guarded transaction and returns every row. + /// + /// A single statement. With SQL rewriting off, Postgres rejects a second one. Parameters are + /// $1, $2 and so on. + /// + /// The parameter values, bound by position and typed by their .NET type. + /// Cancels the query. The rollback still runs. + /// The rows, by column name. SQL NULL is . + /// + /// The statement failed: it tried to write (25006), ran past 5 seconds (57014), waited over + /// 1 second for a lock (55P03), or raised any other error. + /// public async Task>> QueryAsync( string sql, IReadOnlyList parameters, CancellationToken cancellationToken) { @@ -91,6 +113,8 @@ public static async Task OpenAsync(NpgsqlConnectionStringBuilde } } + /// Closes the connection. + /// A task that completes when the connection is closed. public async ValueTask DisposeAsync() { await connection.DisposeAsync(); diff --git a/src/Pgcheckup/Engine/Scanner.cs b/src/Pgcheckup/Engine/Scanner.cs index 2463c84..9df65b0 100644 --- a/src/Pgcheckup/Engine/Scanner.cs +++ b/src/Pgcheckup/Engine/Scanner.cs @@ -3,24 +3,47 @@ namespace Pgcheckup.Engine; +/// What the report's header says about the server. +/// The database scanned. +/// The host as given, never the full connection string. +/// The Postgres version, such as "17.6". public sealed record ServerInfo(string Database, string Host, string Version); +/// One check's findings. +/// The check that ran. +/// What it found. Empty when it passed. public sealed record CheckResult(CheckDefinition Check, IReadOnlyList Findings) { + /// The most severe finding's severity, or when the check passed. public Severity? Worst => Findings.Count == 0 ? null : Findings.Max(f => f.Severity); } +/// Everything a scan found. +/// The server scanned. +/// Each check's findings, in the order the checks ran. public sealed record ScanReport(ServerInfo Server, IReadOnlyList Results); -// A check that couldn't run. In M0 this ends the scan; M1 reports it as errored and goes on. +/// A check that couldn't run. In M0 this ends the scan; M1 reports it as errored and goes on. +/// The check that failed. +/// What went wrong. public sealed class CheckFailedException(string checkId, Exception inner) : Exception($"{checkId} couldn't run: {inner.Message}", inner) { + /// The check that failed. public string CheckId { get; } = checkId; } +/// Runs checks against one database. public static class Scanner { + /// Reads the server's version and database, then runs each check in turn. + /// The guarded session to query through. + /// The host to name in the report. + /// The checks to run, in order. + /// Cancels the scan. + /// The server and each check's findings. + /// A check failed for any reason. The scan stops there. + /// The server's version or database couldn't be read. public static async Task ScanAsync( ReadOnlySession session, string host, IReadOnlyList checks, CancellationToken cancellationToken) { From f467ba965a4d5906ef3dc1dd154be3a385f0168a Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 08:48:30 -0500 Subject: [PATCH 25/26] build: fail the build on missing XML docs --- src/Directory.Build.props | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 src/Directory.Build.props diff --git a/src/Directory.Build.props b/src/Directory.Build.props new file mode 100644 index 0000000..1d13767 --- /dev/null +++ b/src/Directory.Build.props @@ -0,0 +1,9 @@ + + + + + true + + false + + From 4a98cb13d50418809944ceb7699d46e98fe2e969 Mon Sep 17 00:00:00 2001 From: Hazel Granados Date: Fri, 25 Sep 2026 08:48:30 -0500 Subject: [PATCH 26/26] docs(agents): require XML docs on public code --- AGENTS.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 92a4fa5..b8ef622 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,4 +73,5 @@ The product is only as good as these rules. Never break them, not even in debug - **Checks:** automate acceptance checks instead of handing manual steps to the owner. Give every check that tests for an absence a positive control, meaning a case that proves the check can fail. - **Validation:** evidence comes from dogfooding (the log) and public async signals (issues, PRs, downloads, image pulls). Don't plan interviews, recruiting or outreach. - **Docs:** short and concise. Prefer editing `ROADMAP.md` over creating new planning documents. Repo files never reference the owner's private notes. -- **Code comments:** explain why, not what. Only comment on what the code can't say for itself: a non-obvious constraint, a workaround and its cause, or a line that keeps a hard rule. Don't restate names or types, don't add boilerplate XML docs, and don't leave commented-out code. A check's `check.md` is its documentation. +- **XML docs:** every public type and member in `src/` has an XML doc comment (`///`), including new ones. The build enforces it: `src/Directory.Build.props` turns on the doc file, so a missing comment is error CS1591. Say what the member does and its contract: parameters, what it returns, what it throws and edge cases, with `` to related types. Don't just restate the name. Document internal and private members too when their purpose isn't obvious from the name. Tests don't need XML docs, because their names say what they check. +- **Code comments:** inside code, explain why, not what. Only comment on what the code can't say for itself: a non-obvious constraint, a workaround and its cause, or a line that keeps a hard rule. Don't leave commented-out code. A check's `check.md` is its documentation.