diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3316eca..48b9804 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,6 +1,7 @@ name: CI on: + workflow_call: push: branches: [master] pull_request: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c2a9863..b5d1d54 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -22,8 +22,12 @@ permissions: id-token: write jobs: + v1-gate: + uses: ./.github/workflows/ci.yml + publish: name: Publish to npm + needs: v1-gate runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 @@ -40,3 +44,4 @@ jobs: - run: npm ci - run: npm publish --loglevel verbose + working-directory: packages/core diff --git a/.gitignore b/.gitignore index 8042b37..4bd5663 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,4 @@ node_modules/ dist/ -tests/perf/generated/ +tests/performance/generated/ *.log diff --git a/CHANGELOG.md b/CHANGELOG.md index 67c94f0..2b6af76 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,30 @@ Notable changes to this project, following [Keep a Changelog](https://keepachang ## [Unreleased] +### Architecture + +- Removed runtime statement parsing from the `node:sqlite` adapter. Driver column metadata now selects the row path, older `node:sqlite` versions fall back to `all()` without write metadata, and SQLite counters provide write metadata without interpreting user SQL. +- Moved the schema model out of Compiler into its own internal Schema boundary and changed architecture checks from partial forbidden-import rules to an allowlisted dependency graph that also recognizes TypeScript import expressions and CommonJS import-equals declarations. +- Unified public output and parameter inference around one Language parse result. Parameter inference remains schema-aware in Compiler and consumes only fragments already delimited by shallow Query IR. +- Made the release workflow depend on the complete reusable CI gate before publishing. + +### Changed + +- **Breaking (pre-v1):** removed the accidental advanced exports `ParseSelect`, `ParseStatement`, `ParsedStatement`, `Source`, and `FunctionReturnTypes`. They exposed the retired parser and compiler internals and have no supported replacement. Use `Query`, `Row`, `StrictQuery`, `StrictRow`, and `Params` for the public type-inference contract. +- The reviewed v1 type-instantiation baseline is 135,217, down from 244,919 after the Legacy compiler and accidental parser exports were removed. The public hard ceiling is reduced from 250,000 to 150,000; eight isolated stress cases add a 10% relative gate and a 200,000 absolute ceiling. + +### Strict mode + +- Write targets, write columns, DML join predicates, and the SELECT half of `INSERT ... SELECT` now receive the strict checks listed under Fixed. Correlated subqueries that are valid SQL no longer produce an outer-scope error. + +### Type mapping + +- Corrected inferred row mappings for parenthesized columns, function-call projections, set-operation branches, and CTE shadowing are listed under Fixed. `Params` corrections are called out individually because tuple arity and element types are product API. + +### SQL and dialect support + +- The supported union now includes the documented PostgreSQL `DISTINCT ON` spellings, parenthesized set-operation branches, and T-SQL `OUTPUT ... FROM` forms. Dialect capability and placeholder differences are executable in the v1 dialect matrix. + ### Fixed - `WITH t AS MATERIALIZED (...)` and its `NOT MATERIALIZED` twin parse again. The parser wanted the body's opening paren directly after `as`, so the Postgres 12 planner hint took the whole WITH clause down with it and the query degraded into an index signature row instead of reporting anything ([#283](https://github.com/tiagolauer/OwlSQL/issues/283)). @@ -41,7 +65,7 @@ Notable changes to this project, following [Keep a Changelog](https://keepachang ### Changed -- **Breaking:** the editor plugin moved out of this package into its own, [`@owlsql/ts-plugin`](ts-plugin/README.md). The `@owlsql/core/ts-plugin` subpath is gone. To migrate, install `@owlsql/ts-plugin` as a dev dependency and change the plugin name in your `tsconfig.json`: +- **Breaking:** the editor plugin moved out of this package into its own, [`@owlsql/ts-plugin`](packages/ts-plugin/README.md). The `@owlsql/core/ts-plugin` subpath is gone. To migrate, install `@owlsql/ts-plugin` as a dev dependency and change the plugin name in your `tsconfig.json`: ```json { diff --git a/COMPARISON.md b/COMPARISON.md index 9f54c34..3a53385 100644 --- a/COMPARISON.md +++ b/COMPARISON.md @@ -134,16 +134,17 @@ check the source link and open an issue. - **Bundle**: zero runtime dependencies (`package.json` has no `dependencies` field), and the runtime surface is `createTypedDb`, `defineSchema`, and the `Result` helpers — about 175 lines of source - across [`src/index.ts`](src/index.ts) and [`src/result.ts`](src/result.ts) + across [`packages/core/src/index.ts`](packages/core/src/index.ts) and + [`packages/core/src/runtime/result.ts`](packages/core/src/runtime/result.ts) combined, most of which is type declarations erased at compile time. The - parser itself (a few thousand lines across `src/parse.ts`/`src/from.ts`/ - etc.) is 100% types — it ships zero bytes to any runtime. + compiler under `packages/core/src/language` and `packages/core/src/compiler` + is 100% types — it ships zero bytes to any runtime. - **DX trade-off, stated plainly**: this is the smallest surface area of the five because it does the least. No migrations, no relation loading, no query builder ergonomics (autocomplete for chained methods) — you write SQL, you get a type back. If you want an ORM's feature set, this isn't - one; see the [Supported SQL subset](README.md#supported-sql-subset) and - [Limitations](README.md#limitations) for exactly where the parser's + one; see the [Supported SQL subset](packages/core/README.md#supported-sql-subset) and + [Limitations](packages/core/README.md#limitations) for exactly where the parser's coverage ends. ## Methodology notes diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index dbd62d9..2cb23be 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,7 +26,7 @@ Keep a PR to one fix or one feature. A PR that touches three unrelated things is If the change alters what a query infers to, say so in the description and name the bump it implies under [VERSIONING.md](VERSIONING.md). A row shape that gains, loses, or retypes a key is a breaking change even when no runtime signature moved. -Every behavior change needs a test that would fail without the fix. If you're touching `src/parse.ts`, `src/where.ts`, or another type-level file, that usually means a `.test-d.ts` case with `@ts-expect-error` or an `Equal<>` assertion; runtime behavior (adapters, the CLI, the editor plugin) gets a `.test.ts` case instead. A PR without a regression test is a PR someone else will eventually re-break by accident. +Every behavior change needs a test that would fail without the fix. If you're touching `packages/core/src/language`, `packages/core/src/compiler`, or another type-level file, that usually means a `.test-d.ts` case with `@ts-expect-error` or an `Equal<>` assertion; runtime behavior (adapters, the CLI, the editor plugin) gets a `.test.ts` case instead. A PR without a regression test is a PR someone else will eventually re-break by accident. ### Architecture changes @@ -41,7 +41,7 @@ ADRs in `docs/adr/`. A change that supersedes one of those decisions needs a new You'll need Node 20 or later. The `node:sqlite` adapter and the CLI's SQLite introspection need Node 22.5+, since `node:sqlite` is newer than the rest of the runtime surface this library targets. -This repository holds two packages, as an npm workspace: `@owlsql/core` at the root, and the editor plugin in [`ts-plugin/`](ts-plugin/README.md). They are apart because they don't reach the same TypeScript versions — the library type-checks clean on TypeScript 7, while the plugin needs the classic compiler API, which TypeScript 7 does not ship at all. One package can only declare one peer range, and either choice would have been a lie about half the code. +This repository holds two npm workspaces: [`@owlsql/core`](packages/core) and the editor plugin in [`packages/ts-plugin/`](packages/ts-plugin/README.md). They are apart because they don't reach the same TypeScript versions — the library type-checks clean on TypeScript 7, while the plugin needs the classic compiler API, which TypeScript 7 does not ship at all. One package can only declare one peer range, and either choice would have been a lie about half the code. ```bash npm install @@ -57,6 +57,8 @@ npm test --workspace @owlsql/ts-plugin # the editor plugin, independently Nothing in the root `npm test` runs plugin code, and that's deliberate: a release of the library should not be gated on a plugin whose supported TypeScript range is narrower and whose future is upstream's to decide. +`npm run test:perf` compiles the public fixture and each case under `tests/performance/cases` independently. Baseline or ceiling changes must include the measured result and rationale in the same commit. + One wrinkle worth knowing before you touch the tsconfigs: the TypeScript 7 CI job runs `test:types:core` rather than `test:types`, because `tests/cli-codegen-edge.test.ts` uses the compiler API (it parses generated schema output to prove it's syntactically valid) and so cannot run there. That one file lives in `tsconfig.compiler-api-tests.json` so it can be left out of the TypeScript 7 run without being quietly dropped from every other one. ### Fixing a bug @@ -70,7 +72,7 @@ Open an issue before writing the implementation if the feature touches the publi ### Design preferences - No runtime SQL parsing, ever. If a change needs to inspect the query string at runtime to work, it probably belongs in the ts-plugin (which already does its own lightweight runtime scanning for editor support), not in the core library. -- Adapters (`src/adapters/*.ts`) import the driver's types only, never the driver package itself as a value. This keeps `@owlsql/core/pg` usable without `pg` actually being installed, for anyone who only imports a different adapter. +- Adapters (`packages/core/src/adapters/*.ts`) import the driver's types only, never the driver package itself as a value. This keeps `@owlsql/core/pg` usable without `pg` actually being installed, for anyone who only imports a different adapter. - If you extend the SQL subset the parser accepts, update the "Supported SQL subset" and "Limitations" sections in the README in the same PR. A parser change nobody can discover from the docs is half a feature. - Prefer a documented scope boundary over a half-correct implementation. Several existing features (LATERAL correlation, WHERE-clause diagnostics with parens) deliberately do less than a full SQL engine would, and say so in the README, rather than guessing. @@ -78,8 +80,8 @@ Open an issue before writing the implementation if the feature touches the publi Three layers, and they test different things: -- **Type tests** (`tests/*.test-d.ts`) are pure type assertions. If they compile, the inference is correct; there's no runtime assertion to run. They cover column/alias projection, `@ts-expect-error` cases for queries that should fail to type, permissive-inference locks, and deep-recursion stress. -- **Runtime tests** (`tests/*.test.ts`) run under vitest and cover the executor/`Result` contract, adapter parameter handling, and the CLI. Drivers are faked here, so these prove the adapter's own logic, not what a real server sends back. The plugin's own tests live beside it in `ts-plugin/tests/`. +- **Type tests** (`packages/core/tests/*.test-d.ts`) are pure type assertions. If they compile, the inference is correct; there's no runtime assertion to run. They cover column/alias projection, `@ts-expect-error` cases for queries that should fail to type, permissive-inference locks, and deep-recursion stress. +- **Runtime tests** (`packages/core/tests/*.test.ts`) run under vitest and cover the executor/`Result` contract, adapter parameter handling, and the CLI. Drivers are faked here, so these prove the adapter's own logic, not what a real server sends back. The plugin's own tests live in `packages/ts-plugin/tests/`. - **Integration tests** (`tests/integration/*.test.ts`) run the adapters and the `generate` CLI against real PostgreSQL, MySQL, and SQL Server instances. They cover what a fake driver can't: how each driver actually decodes a column (`bigint`, `numeric`, `tinyint(1)`, `bit`), the metadata a real result carries, and whether a rolled-back transaction really left no rows behind. CI runs the type tests against a matrix of TypeScript versions, since a template-literal-type change that works on one TypeScript release can silently stop working (or start working differently) on another. @@ -128,7 +130,7 @@ npm version git push --follow-tags origin master ``` -Then run **Actions → Release → Run workflow**, or publish a GitHub Release for the tag — either triggers it. The workflow runs `npm publish --provenance`, and `prepublishOnly` puts the type tests, the runtime tests and the build in front of that, so a red build cannot reach the registry. +Then run **Actions → Release → Run workflow**, or publish a GitHub Release for the tag — either triggers it. The release calls the same reusable CI workflow used by pushes and pull requests before publishing, including the TypeScript and Node matrices, plugin, architecture, package, performance, and real-database integration gates. `prepublishOnly` repeats the core type, runtime, architecture, and build checks immediately before npm receives the package. This needs the package's *Trusted publisher* to be configured once on npmjs.com (package → Settings → Trusted publisher → GitHub Actions, repository `tiagolauer/OwlSQL`, workflow `release.yml`). diff --git a/README.md b/README.md index 92523c9..28c9333 100644 --- a/README.md +++ b/README.md @@ -1,1088 +1,3 @@ # OwlSQL -> Write raw SQL. Get fully-typed results. No codegen, no ORM, no runtime parsing. - -OwlSQL (`@owlsql/core`) reads your SQL inside TypeScript's type system and -infers the row shape from the query string and your schema. It happens as you -type, in your editor. There is no build step. - -```ts -type DB = { - users: { id: number; name: string; email: string; active: boolean }; -}; - -const db = createTypedDb(createPgExecutor(pool)); - -const a = await db.query('select id from users'); -// a.value ^? { id: number }[] - -const b = await db.query('select name as handle, active from users'); -// b.value ^? { handle: string; active: boolean }[] - -const c = await db.query('select * from users'); -// c.value ^? { id: number; name: string; email: string; active: boolean }[] - -const d = await db.query('select id from users where id = $1', 7); -// ^ typed as number -``` - -Rename a column in the SQL, mistype a field, or select something that does not -exist, and the result type changes immediately, before you run a single line. -There is **no generated file to keep in sync** and **no SQL parser shipped to -production**: all the work happens during type checking. - -It is **not** an ORM and **not** a query builder. It does not connect to your -database. You keep writing the SQL you already know; this library only layers -compile-time result typing on top of whatever driver you use. - -**[Try it in your browser →](https://stackblitz.com/github/tiagolauer/OwlSQL/tree/master/examples/playground?file=index.ts)** -No install, no database — see [`examples/playground`](examples/playground). - ---- - -## Table of contents - -- [The problem](#the-problem) -- [How it works](#how-it-works) -- [Install](#install) -- [How it compares](#how-it-compares) -- [What it costs to compile](#what-it-costs-to-compile) -- [Tutorial](#tutorial) - - [1. Describe your schema](#1-describe-your-schema) - - [2. Create a typed client](#2-create-a-typed-client) - - [3. Run queries and handle the Result](#3-run-queries-and-handle-the-result) - - [4. Aliases, `*`, and qualified columns](#4-aliases--and-qualified-columns) - - [5. Type-only usage (no client)](#5-type-only-usage-no-client) - - [6. Aggregates and functions](#6-aggregates-and-functions) - - [7. INSERT / UPDATE / DELETE with RETURNING](#7-insert--update--delete-with-returning) - - [8. Strict mode — turn typos into type errors](#8-strict-mode--turn-typos-into-type-errors) - - [9. Joins](#9-joins) - - [10. Typed parameters](#10-typed-parameters) - - [11. Transactions](#11-transactions) -- [Driver recipes](#driver-recipes) -- [Database support](#database-support) -- [Editor autocomplete](#editor-autocomplete) -- [API reference](#api-reference) -- [Supported SQL subset](#supported-sql-subset) -- [Limitations](#limitations) -- [FAQ](#faq) -- [Contributing](#contributing) -- [License](#license) - ---- - -## The problem - -I was building a TypeScript backend and picked raw SQL over an ORM on purpose. -I wanted control over the queries and no layer of magic between my code and the -database. That part worked. - -The return types were the problem. Every query came back as `any[]` or -`unknown[]`, so I wrote an interface by hand for each one: - -```ts -interface UserListRow { id: number; name: string } -const rows = (await pool.query('select id, name from users')).rows as UserListRow[]; -``` - -Those interfaces drift. Someone adds `email` to the SQL, forgets the interface, -and the type quietly lies until it breaks in production. They are also -boilerplate: the interface restates the query in a second syntax, so you type -the same column list twice. - -The usual fixes each cost something. ORMs replace your SQL with their own DSL -and runtime, which was the thing I was trying to avoid. Codegen tools do give -you accurate types, but they bolt a generation step onto the build, so now you -have a watcher, a CLI, a database connection at build time, and generated files -in version control. - -The query string is already the source of truth, so the compiler may as well -read it. TypeScript's template literal types can parse a `SELECT` and map its -columns to a schema during type checking, which is what this library does. - -## How it works - -There is no runtime SQL parser and no build step. The entire parser is written -as recursive [template literal types](https://www.typescriptlang.org/docs/handbook/2/template-literal-types.html) -evaluated by `tsc`: - -1. **Normalize** — collapse newlines, tabs, and runs of spaces into a single - trimmed, single-spaced string. -2. **Parse** — strip the `SELECT` keyword, split on the first case-insensitive - `FROM`, and separate the column list from the table name. -3. **Resolve** — parse each column into `[outputName, sourceColumn]` (handling - `AS` aliases and `table.col` qualifiers), then look the column up in your - schema to get its TypeScript type. -4. **Assemble** — build `{ ...columns }[]`. - -The JavaScript that actually ships is a tiny passthrough: it forwards your SQL -to the driver you provide and wraps the rows in a `Result`. All the intelligence -lives in the `.d.ts` types. - -## Install - -```bash -npm install @owlsql/core -``` - -`typescript` is a peer dependency (**>= 5.4, < 8** — 5.4 is the oldest -version CI tests; the ts-plugin does not load on TS 7). You almost -certainly already have it. - -The package is **ESM-only** (`import` only — `require()` is not supported). -Node support: **>= 20** for the library and CLI; the `node:sqlite` adapter -and the CLI's SQLite introspection additionally need **Node >= 22.5** (they -fail with a clear error below that). - -## How it compares - -| | OwlSQL | Prisma | Kysely | pgTyped | Zapatos | -| --- | --- | --- | --- | --- | --- | -| You write | Raw SQL strings | Prisma's own query API | Builder method chains | Raw SQL in `.sql` files or tags | Helpers, or raw SQL via `db.sql` | -| Build step | No (opt-in `generate` for the schema only) | Yes, `prisma generate` | No (optional `kysely-codegen`) | Yes, a CLI run against a live database | No (opt-in schema generation) | -| Runtime query engine | None. Your string reaches the driver unchanged | Yes, a TypeScript query compiler | Yes, compiles the chain to SQL on every call | Minimal. Runs a query the CLI already extracted | Yes, builds SQL from helper calls | -| Bundle (min/gzip) | No dependencies; ~175 lines of glue, the parser costs 0 bytes | ~1.6 MB / ~600 KB | 189 KB / 38.7 KB | 399 KB / 85 KB | No bundled engine beyond thin helpers | - -Kysely is the closest of these in spirit: no magic, and inference that goes all -the way down. What differs is what you type. Its builder is a fluent API; here -you type SQL. If you want to paste a query straight out of `psql` or a -migration file and have it work, that is raw SQL, and that is the premise. - -This is not a runtime-speed comparison, on purpose. Your database and driver -dominate query execution, not the layer sitting on top of them, and these five -tools have architectures too different for a queries-per-second figure to say -anything. [COMPARISON.md](COMPARISON.md) has the long version, every number -sourced. - -## What it costs to compile - -Every tool in that table charges you something. Codegen tools charge a build -step, ORMs charge bundle size and a runtime engine, and this one charges -compile time. So here is the number. - -A fixture of 100 tables with 13 columns each, plus 32 queries covering joins, -`GROUP BY`, `CASE`, CTEs, `UNION`, strict mode and typed parameters, -type-checks in: - -| | | -| --- | --- | -| Type instantiations | 166,512 | -| Check time | ~0.4s | -| Runtime cost | 0. Nothing parses SQL at request time | - -Measured with `tsc --extendedDiagnostics` on TypeScript 5.9.3. The cost grows -linearly on top of a fixed overhead: about 96,000 instantiations go to the -schema itself, then roughly 2,200 per query. CI holds that number to a ceiling, -so a change that makes the parser work harder for the same answer fails the -build instead of quietly slowing down every editor that opens your project. - -## Tutorial - -### 1. Describe your schema - -A schema is just a type: table name → column name → TypeScript type. Use a -`type` or an `interface`, whichever you prefer. - -```ts -type DB = { - users: { - id: number; - name: string; - email: string; - active: boolean; - }; - posts: { - id: number; - title: string; - user_id: number; - published: boolean; - }; -}; -``` - -This type is the single source of truth for what your tables look like. It has -no runtime cost — it is erased during compilation. Mark nullable columns with -`| null` (e.g. `bio: string | null`) and that nullability flows straight into -your query results. - -**Optional: generate a starting point with `owlsql generate`.** -Writing that type by hand is fine for a handful of tables, but you can also -have it generated from a real database: - -``` -npx @owlsql/core generate --url postgres://user:pass@host/db --out schema.ts -``` - -This connects to your database, introspects the tables/columns/nullability, -and writes a `schema.ts` with `export interface DB { ... }` — the exact shape -from step 1 above. It's a **one-shot generator, not a codegen pipeline**: the -library still parses your queries entirely at the type level with zero -runtime codegen, same as always. The generated file is a normal `.ts` file — -commit it, edit it by hand afterward, rename fields, anything. Running -`generate` again just overwrites it with a fresh snapshot; nothing stays -"synced" automatically — unless you opt into checking for that in CI with -`--check` (below). - -| Flag | Required | Description | -| ---- | -------- | ----------- | -| `--url` | yes | Connection string (or a file path for SQLite). SQL Server accepts both `mssql://user:pass@host:1433/db` (translated to a driver config; `?encrypt=false` and `?trustServerCertificate=true` supported, and a named instance may be written as `host\INSTANCE`) and an ADO string (`Server=host;Database=db;User Id=u;Password=p`). | -| `--out` | no | Output file. Defaults to `./schema.ts`. | -| `--dialect` | no | `postgres` \| `mysql` \| `sqlite` \| `mssql`. Auto-detected from the URL scheme (`postgres://`/`postgresql://`, `mysql://`, `mssql://`/`sqlserver://`); an ADO `Server=...` string also routes to `mssql` — falls back to `sqlite` for a bare file path, so it's only needed when that's ambiguous. | -| `--schema` | no | Schema/database name to introspect. Defaults to `public` (Postgres), the connected database (MySQL), or `dbo` (SQL Server). Not used for SQLite. | -| `--table` | no | Comma-separated list (`--table users,posts`). Only introspect these tables, instead of every table in the schema. | -| `--exclude` | no | Comma-separated list. Skip these tables even if `--table` would otherwise include them. | -| `--check` | no | Don't write `--out` — introspect and render as usual, then compare against the existing file. Exits `0` with no output if they match, `1` with a message telling you where they first differ (or that the file doesn't exist yet) if they don't. `--table`/`--exclude`/`--schema` apply identically, so the comparison stays meaningful. Useful in CI to catch a migration that ran without anyone regenerating the committed schema. | - -```bash -# CI: fail the build if schema.ts has drifted from the real database -npx @owlsql/core generate --url "$DATABASE_URL" --out schema.ts --check -``` - -`generate` needs the matching driver installed as a real dependency (`pg`, -`mysql2`, or `mssql` — SQLite uses the `node:sqlite` builtin, Node ≥22.5). It -prints a clear error telling you which one to install if it's missing. - -**Type mapping follows each driver's defaults.** `pg` hands back `bigint`, -`numeric`/`decimal` and `money` as `string`; `mysql2` returns `decimal` as -`string` but `bigint` as a JS `number` (unless you enable -`supportBigNumbers`/`bigNumberStrings`); `mssql` (tedious) returns `bigint` as -`string` but parses `decimal`/`numeric`/`money` into JS `number` (with -precision loss beyond 2^53). SQLite has no column types, only affinities, so -the *declared* type drives the mapping: `INTEGER`/`REAL` and the numeric -names (`NUMERIC`, `DECIMAL(10,2)`, `MONEY`) become `number`, text-affinity -types and `JSON` become `string`, `BLOB` and an untyped column become -`Uint8Array`, and a declared type that says nothing about its contents -(`GEOMETRY`, a custom name) becomes `unknown` rather than a guess. The -generated types mirror exactly that. If your driver is configured -differently, just edit the generated field by hand; it's a plain type after -that point. - -### 2. Create a typed client - -The library never touches your database. You hand `createTypedDb` an -**executor**: a function that takes `(sql, params)`, runs it against your real -driver, and returns the raw rows. - -```ts -import { Pool } from 'pg'; -import { createTypedDb } from '@owlsql/core'; - -const pool = new Pool(); - -const db = createTypedDb(async (sql, params) => { - const res = await pool.query(sql, params as unknown[]); - return res.rows; -}); -``` - -`db` is now bound to your schema. Every query you run through it will be typed -against `DB`. - -### 3. Run queries and handle the Result - -`query` does not throw on failure. It returns a **`Result`** — a discriminated -union of success or error — so failures are values you handle explicitly. - -```ts -import { ResultStatus } from '@owlsql/core'; - -const result = await db.query('select id, name from users'); - -if (result.status === ResultStatus.Error) { - console.error(result.error.kind, result.error.message); - return; -} - -result.value; -// ^? { id: number; name: string }[] -for (const user of result.value) { - console.log(user.id, user.name); -} -``` - -Prefer a helper over the `status` field? `isOk` / `isErr` narrow the same way: - -```ts -import { isOk } from '@owlsql/core'; - -const result = await db.query('select id, email from users'); - -if (isOk(result)) { - result.value; - // ^? { id: number; email: string }[] -} -``` - -> ⚠️ **Pass the SQL as a string literal**, not a `string` variable. If the type -> widens to `string`, the compiler can no longer see the query and inference -> falls back to `unknown`. `db.query('select id from users')` ✅ — -> `const q: string = ...; db.query(q)` ❌. - -### 4. Aliases, `*`, and qualified columns - -```ts -const renamed = await db.query('select id, name as username from users'); -// renamed.value ^? { id: number; username: string }[] - -const implicit = await db.query('select name handle from users'); -// implicit.value ^? { handle: string }[] - -const qualified = await db.query('select u.id, u.name from users u'); -// qualified.value ^? { id: number; name: string }[] - -const everything = await db.query('select * from users'); -// everything.value ^? { id: number; name: string; email: string; active: boolean }[] -``` - -Trailing clauses are ignored for inference — they do not change the row shape: - -```ts -const recent = await db.query( - 'select id, title from posts where published = true order by id limit 10', -); -// recent.value ^? { id: number; title: string }[] -``` - -Keywords are case-insensitive and whitespace/newlines are tolerated, so -formatted multi-line queries work as-is: - -```ts -const r = await db.query(` - SELECT id, - title - FROM posts - WHERE published = true -`); -// r.value ^? { id: number; title: string }[] -``` - -### 5. Type-only usage (no client) - -Sometimes you only want the *type* of a query — for an API contract, a DTO, or a -function signature — without running anything. Use the `Query` type directly: - -```ts -import type { Query } from '@owlsql/core'; - -type UserListRow = Query; -// ^? { id: number; email: string }[] - -function renderUsers(rows: Query) { - // rows is { id: number; name: string }[] -} -``` - -`Row` gives the single-row object (without the surrounding array) if you -need it. - -### 6. Aggregates and functions - -Common SQL functions resolve to their return type, and the output column is -named after the function (or its alias): - -```ts -const stats = await db.query('select count(*) from users'); -// stats.value ^? { count: number }[] - -const named = await db.query('select count(*) as total, max(age) as oldest from users'); -// named.value ^? { total: number; oldest: number }[] - -const shout = await db.query('select id, upper(name) as name from users'); -// shout.value ^? { id: number; name: string }[] -``` - -Recognized: `count`, `sum`, `avg`, `min`, `max`, `length`, `char_length`, -`octet_length`, `abs`, `ceil`, `floor`, `round`, `power`, `mod`, `greatest`, -`least`, `row_number`, `rank`, `dense_rank`, `ntile`, `percent_rank`, -`cume_dist` → `number`; `lower`, `upper`, `trim`, `ltrim`, `rtrim`, `concat` → -`string`; `coalesce`, `nullif`, `lag`, `lead`, `first_value`, `last_value`, -`nth_value` → `unknown`; `now`, `current_timestamp`, `current_date` → `Date`. -Anything else resolves to `unknown`. This return-type table is -dialect-agnostic, which isn't always what the driver actually hands back for -`count`/`sum`/`avg` — see [Limitations](#limitations). - -### 7. INSERT / UPDATE / DELETE with RETURNING - -`RETURNING` is typed exactly like a `SELECT` projection against the target -table: - -```ts -const created = await db.query( - 'insert into users (name, email) values ($1, $2) returning id, name', -); -// created.value ^? { id: number; name: string }[] - -const updated = await db.query('update users set active = $1 where id = $2 returning *'); -// updated.value ^? { id: number; name: string; email: string; active: boolean }[] -``` - -A write without `RETURNING` resolves to `Record[]` (no row -columns). - -### 8. Strict mode — turn typos into type errors - -By default an unknown column or table resolves to `unknown` (permissive). Pass -`{ strict: true }` and the result instead becomes a `QueryTypeError` carrying a -human-readable message, so a typo is impossible to ignore: - -```ts -const db = createTypedDb(executor, { strict: true }); - -const ok = await db.query('select id, name from users'); -// ok.value ^? { id: number; name: string }[] - -const typo = await db.query('select naem from users'); -// typo.value ^? QueryTypeError<'unknown column: naem'>[] -``` - -The error type propagates wherever you use the rows, surfacing the message in -hovers and breaking any code that treats them as real data. - -Strict mode checks the `SELECT` list, the `WHERE` clause, and `JOIN ... ON` -conditions — including the `WHERE` of an `UPDATE`/`DELETE` that returns no -columns, where a typo is most expensive: - -```ts -const wrongSide = await db.query( - 'select u.id from users u join orders o on u.id = o.id', -); -// wrongSide.value ^? QueryTypeError<'unknown column: id'>[] (when orders has no such column) -``` - -`GROUP BY`, `HAVING`, and `ORDER BY` are **not** checked — they have their own -resolution rules (a `SELECT`-list alias, an ordinal, an aggregate), so a name -there is not necessarily a column of a source table. - -### 9. Joins - -`INNER`, `LEFT`, `RIGHT`, `FULL` (with optional `OUTER`), and `CROSS` joins are -supported, with table aliases and any number of joins. Qualified columns -(`alias.column`) resolve to the aliased table; unqualified columns are searched -across every joined table. `alias.*` expands one table; a bare `*` expands all. - -```ts -const rows = await db.query( - 'select u.name, p.title from users u join posts p on u.id = p.user_id', -); -// rows.value ^? { name: string; title: string }[] -``` - -An outer join makes the optional side's columns nullable: `LEFT` nulls the -right-hand table, `RIGHT` nulls the left-hand table, and `FULL` nulls both. - -```ts -const rows = await db.query( - 'select u.name, p.title from users u left join posts p on u.id = p.user_id', -); -// rows.value ^? { name: string; title: string | null }[] -``` - -`select *` across a join merges the columns of every table (applying join -nullability). In strict mode, an unknown alias becomes -`QueryTypeError<'unknown alias: x'>`. - -### 10. Typed parameters - -Placeholders in the query are typed from the column they're compared against, so -`query` checks the **number and types** of the arguments you pass: - -```ts -await db.query('select id from users where id = $1', 1); -// ^ inferred [number] - -await db.query('select id from users where id = $1 and name = $2', 1, 'ada'); -// inferred [number, string] - -// @ts-expect-error wrong type — id is a number -await db.query('select id from users where id = $1', 'oops'); - -// @ts-expect-error wrong count — one param expected -await db.query('select id from users where id = $1'); -``` - -Both numbered (`$1`, `$2`) and positional (`?`) placeholders work, including -across joins (`where p.views > $1` resolves against the aliased table). Use the -`Params` type to get the tuple on its own. - -For this to work, write the comparison **with spaces around the operator** -(`id = $1`, not `id=$1`) — that is what lets the compiler see the column, -operator, and placeholder as separate tokens. - -**Placeholder-style checking (opt-in).** The type layer accepts `$n`, `?` and -`@name` interchangeably, but each driver only understands its own style — `?` -with the pg adapter is a runtime syntax error. Declare the style your executor -expects and mismatches become compile errors: - -```ts -const db = createTypedDb(createPgExecutor(pool)); - -// @ts-expect-error '?' is not a pg placeholder — use $1 -await db.query('select id from users where id = ?', 1); -``` - -Styles: `'dollar'` (pg, postgres.js), `'question'` (mysql2), `'at'` (mssql). -`node:sqlite` accepts all three plus `:name`, so leave the option off there. -A `:name` placeholder is typed like any other but carries no style of its own, -so it is never checked against a declared dialect. - -**Write metadata.** Adapters report driver metadata alongside the rows: on a -successful `Result`, `result.meta?.rowCount` carries the affected-row count -and `result.meta?.lastInsertRowid` the generated id (where the driver -provides one), so an INSERT without `RETURNING` is no longer a black box. - -### 11. Transactions - -There is a footgun to know about: **never run `BEGIN`/`COMMIT` through an -executor bound to a pool.** Each `query()` may check out a *different* -connection, so `BEGIN` runs on connection A and `COMMIT` on connection B, -leaving an open transaction (and its locks) on a pooled connection that is -later handed to another caller. - -`pg`, `postgres.js`, `mysql2`, and `mssql` each ship a small transaction -helper that pins one connection for the whole callback and handles -begin/commit/rollback for you: - -```ts -import { Pool } from 'pg'; -import { createPgTransaction } from '@owlsql/core/pg'; - -const pool = new Pool(); - -async function transferFunds(from: number, to: number, amount: number) { - await createPgTransaction(pool)(async (tx) => { - await tx.query('update accounts set balance = balance - $1 where id = $2', amount, from); - await tx.query('update accounts set balance = balance + $1 where id = $2', amount, to); - }); -} -``` - -`createPgTransaction(pool)` returns the function that actually runs the -transaction — it's curried on `DB` because TypeScript can't partially infer -type arguments; a single `createPgTransaction(pool, fn)` call would -compile, but would silently stop inferring the callback's return type and -type it `unknown` instead. Splitting `DB` into its own call keeps the second -call (`(fn, options?)`) argument-only, so both the optional `options` and the -callback's return type infer normally. - -The callback's `tx` is a full `TypedDb`, typed exactly like the one -`createTypedDb` returns (pass `{ strict: true }` as the second argument to -the inner call the same way: `createPgTransaction(pool)(fn, { strict: -true })`). The transaction commits if the callback resolves and rolls back if -it throws — a rejected `tx.query()` result (the normal `Result` error path) -does *not* trigger a rollback by itself, only a thrown error does, same as -everywhere else this library never throws on a query failure. - -`createMysql2Transaction(pool)(fn, options?)` and -`createMssqlTransaction(pool)(fn, options?)` work the same way. -`createPostgresJsTransaction(sql)(fn, options?)` wraps postgres.js's own -`sql.begin(...)`, which already pins the connection and handles -commit/rollback itself. - -Kysely users should use Kysely's own `db.transaction()`. `node:sqlite` is a -single connection, so plain `begin`/`commit` statements are safe there and no -helper is provided. - -If the rollback *itself* fails, the helper throws an `AggregateError` whose -`errors[0]` is the original failure and `errors[1]` is the rollback failure — -the error that caused the transaction to be abandoned is never replaced by a -cleanup error. - -Under the hood, each helper does exactly what you'd otherwise write by hand: - -```ts -const client = await pool.connect(); -const tx = createTypedDb(createPgExecutor(client), { placeholders: 'dollar' }); - -try { - await client.query('begin'); - await tx.query('update accounts set balance = balance - $1 where id = $2', amount, from); - await tx.query('update accounts set balance = balance + $1 where id = $2', amount, to); - await client.query('commit'); -} catch (error) { - await client.query('rollback'); - throw error; -} finally { - client.release(); -} -``` - -## Driver recipes - -The executor is the only thing that touches your database, so any driver -works. For the most common drivers, OwlSQL ships a ready-made -adapter — import it from its own subpath and pass your existing client -straight in. No dependency is pulled in unless you import that specific -subpath (each driver is an optional peer dependency). - -**node-postgres (`pg`)** - -```ts -import { Pool } from 'pg'; -import { createPgExecutor } from '@owlsql/core/pg'; - -const db = createTypedDb(createPgExecutor(new Pool())); -``` - -**mysql2** - -```ts -import { createPool } from 'mysql2/promise'; -import { createMysql2Executor } from '@owlsql/core/mysql2'; - -const db = createTypedDb(createMysql2Executor(createPool({ /* ... */ }))); -``` - -**postgres.js** - -```ts -import postgres from 'postgres'; -import { createPostgresJsExecutor } from '@owlsql/core/postgres'; - -const db = createTypedDb(createPostgresJsExecutor(postgres())); -``` - -**node:sqlite** (Node's built-in SQLite module, no dependency to install — Node ≥22.5) - -```ts -import { DatabaseSync } from 'node:sqlite'; -import { createNodeSqliteExecutor } from '@owlsql/core/node-sqlite'; - -const db = createTypedDb(createNodeSqliteExecutor(new DatabaseSync('app.db'))); -``` - -**better-sqlite3** (synchronous driver wrapped in a promise — no dedicated -adapter, the same one-liner works with `node:sqlite`'s adapter since both -expose `prepare(sql).all(...params)`) - -```ts -import Database from 'better-sqlite3'; -const sqlite = new Database('app.db'); -const db = createTypedDb(async (sql, params) => sqlite.prepare(sql).all(...params)); -``` - -**Kysely** - -```ts -import { Kysely, PostgresDialect } from 'kysely'; -import { createKyselyExecutor } from '@owlsql/core/kysely'; - -const kysely = new Kysely({ dialect: new PostgresDialect({ /* ... */ }) }); -const db = createTypedDb(createKyselyExecutor(kysely)); -``` - -The adapter runs your query through `CompiledQuery.raw`, which forwards the -SQL text and parameters straight to the underlying driver with **no -placeholder translation** — the SQL you write still has to use whichever -placeholder syntax your configured dialect's own driver expects (`$1` for -`PostgresDialect`, `?` for `MysqlDialect`/`SqliteDialect`). What the adapter -does *not* care about is which Kysely dialect object you passed in; it just -relays whatever string you give it. - -If you want the same compile-time protection against using the wrong -placeholder style that the other adapters get, pass `placeholders` to -`createTypedDb` the same way you would for any of them — it's driven by -that option, not by which adapter produced the executor: - -```ts -const db = createTypedDb(createKyselyExecutor(kysely)); -``` - -**Drizzle (raw SQL)** - -Drizzle's own `sql.raw()` doesn't take a separate parameters array, so it -can't be wired directly into an `Executor`. Instead, reach through Drizzle to -the underlying driver client with [`db.$client`](https://orm.drizzle.team/docs/connect-overview) -and reuse the matching adapter above — one extra line over the plain driver: - -```ts -import { drizzle } from 'drizzle-orm/node-postgres'; -import { createPgExecutor } from '@owlsql/core/pg'; - -const drizzleDb = drizzle(process.env.DATABASE_URL!); -const db = createTypedDb(createPgExecutor(drizzleDb.$client)); -``` - -Swap `createPgExecutor` for `createMysql2Executor`/`createPostgresJsExecutor`/ -`createNodeSqliteExecutor` depending on which Drizzle driver you're using — -`$client` is always the native driver instance underneath. - -**mssql (SQL Server)** - -```ts -import sql from 'mssql'; -import { createMssqlExecutor } from '@owlsql/core/mssql'; - -const pool = await sql.connect({ /* ... */ }); -const db = createTypedDb(createMssqlExecutor(pool)); -``` - -The adapter scans the query for `@name` placeholders (skipping string -literals and `@@` system variables) and binds each one by name via -`request.input(...)`, in order of first appearance — matching how -`Params` types the positional tuple. A repeated `@name` binds once. - -## Database support - -The parser accepts the SQL used by each of the four major engines, without any -per-dialect configuration — it stays permissive and recognizes each dialect's -syntax by shape, not by a declared "mode". - -| Feature | PostgreSQL | MySQL | SQLite | SQL Server | -| ------- | ---------- | ----- | ------ | ---------- | -| Placeholders | `$1`, `$2`, ... | `?` | `?` | `@name`, `@p1` | -| Quoted identifiers | `"col"` | `` `col` `` | `"col"` | `[col]`, `"col"` | -| Row-returning writes | `RETURNING col` | *(not supported by the engine — `INSERT`/`UPDATE`/`DELETE` type as `Record[]`)* | `RETURNING col` | `OUTPUT inserted.col` / `OUTPUT deleted.col` | -| Pagination | `LIMIT n OFFSET m` | `LIMIT n OFFSET m` | `LIMIT n OFFSET m` | `TOP n`, `TOP (n) PERCENT`, or `OFFSET ... FETCH NEXT n ROWS ONLY` | -| `ILIKE` | ✓ | — | — | — | -| Joins, CTEs, `CASE`, window functions, subqueries in `FROM` | ✓ | ✓ | ✓ | ✓ (dialect-agnostic — see [Supported SQL subset](#supported-sql-subset)) | - -See [`tests/dialect-postgres.test-d.ts`](tests/dialect-postgres.test-d.ts), -[`tests/dialect-mysql.test-d.ts`](tests/dialect-mysql.test-d.ts), -[`tests/dialect-sqlite.test-d.ts`](tests/dialect-sqlite.test-d.ts), and -[`tests/dialect-mssql.test-d.ts`](tests/dialect-mssql.test-d.ts) for the exact -query shapes each engine is tested against. - -## Editor autocomplete - -```ts -db.query(` - select id, na -`) -// ^ autocomplete suggests `name` - -db.query(`select id, name from users`) -// ^ hovering shows (column) name: string - -db.query(`select id from users where na`) -// ^ autocomplete suggests `name` -``` - -Want to see it running for yourself before there's a recorded demo here? -[`examples/ts-plugin-demo`](examples/ts-plugin-demo) is a ready-to-open -VSCode project set up for exactly that. - -`@owlsql/ts-plugin` is a **TypeScript Language Service Plugin** — -it runs inside `tsserver`, the same process that already powers VSCode's -IntelliSense, and adds column-name completions while you're still typing the -query string. This is a genuinely different mechanism from the rest of the -library: everything else works by *type-checking* a finished query string; -this works by hooking into the editor's completion request for a string -that isn't even valid SQL yet. - -**Setup** — it ships as its own package, so install it first. - -> **Not on npm yet.** `@owlsql/ts-plugin` has not been published; the -> command below will be the install once it is. Until then, build it from a -> clone of this repository and install that folder: -> -> ```bash -> git clone https://github.com/tiagolauer/OwlSQL -> cd OwlSQL && npm install && npm run build --workspace @owlsql/ts-plugin -> cd /path/to/your/project -> npm install --save-dev /path/to/OwlSQL/ts-plugin -> ``` -> -> Everything below this box is the same either way. - -```bash -npm install --save-dev @owlsql/ts-plugin -``` - -Then add it to your `tsconfig.json`: - -```json -{ - "compilerOptions": { - "plugins": [{ "name": "@owlsql/ts-plugin" }] - } -} -``` - -Then, in VSCode, open the Command Palette and run **"TypeScript: Select -TypeScript Version" → "Use Workspace Version"**. This step is not optional — -VSCode's *bundled* TypeScript does not load workspace plugins, so skipping it -is the #1 reason this kind of plugin appears to do nothing. Other editors -that talk to `tsserver` (Cursor, some Neovim/Sublime LSP setups) generally -pick up `tsconfig.json` plugins automatically. - -**What it does:** suggests column names right after `SELECT`/a comma in the -column list or after `WHERE`/`AND`/`OR`, suggests table names right after -`FROM`/`JOIN` (or a comma in an old-style comma-joined `FROM` list), and -shows a column's resolved type on hover, for `db.query(...)` calls made -through a client built with `createTypedDb`. It is `JOIN`/alias-aware: -every table introduced by a `FROM` or `JOIN` in the same string is in scope, -and typing an alias qualifier (`u.` in `... from users u`) narrows -completions and hover to that one source. With no qualifier, completions/hover -union columns across all sources present so far — the deduplicated union of -every table in `DB` before any `FROM` is typed at all, exactly what covers -the example above. `WHERE`-position completions require a `FROM` to already -be present (there's no table to scope to otherwise); table-name completions -after `FROM`/`JOIN` suggest every table in `DB`, filtered by whatever prefix -you've typed. It also reports unknown columns, unknown tables, unknown -aliases, and ambiguous unqualified columns (present in more than one joined -table) as live editor diagnostics in the `SELECT` list, `FROM`/`JOIN` clause, -and simple `WHERE` comparisons (`where naem = 'x'` squiggles `naem` the -moment you type it) — the same checks strict mode (`{ strict: true }`) -applies at compile time, surfaced as a squiggle while you type instead of -only once the query is finished. - -**What it does not do** (documented scope, not bugs): - -- **`WHERE`-clause diagnostics cover simple comparisons only.** A column - token immediately before `=`/`<>`/`<`/`>`/`<=`/`>=`/`LIKE`/`ILIKE`/`IN`/ - `BETWEEN`/`IS`, or `AND`/`OR`, or the end of the clause, is checked exactly - like a `SELECT`-list column. The moment a `WHERE` clause contains any `(` - or `)` at all — a subquery, a function call, a parenthesized group — the - whole clause is skipped rather than risked: no diagnostics for it, never a - wrong one. `HAVING`/`ORDER BY`/`GROUP BY` aren't checked at all. -- The first `FROM ` is found with a regex, not a real SQL parser: a - `FROM (subquery)` can make it lock onto a table name from inside the - subquery instead of recognizing there's no real outer table yet. -- Only plain string/template literals with **no interpolation** - (`` db.query(`select ...`) ``) are recognized — which is the only form the - library ever expects you to write, since parameters are SQL placeholders - (`$1`/`?`/`@name`), never JS template interpolation. Interpolating - (`` db.query(`select ... ${x}`) ``) silently turns completions/hover off - for that call — there's no squiggle or warning telling you why. -- Completions after `ORDER BY`/`GROUP BY`/`HAVING`/etc. aren't offered yet — - only the `SELECT` column list, `WHERE` clause, and `FROM`/`JOIN` table - names. -- **Requires TypeScript < 7**, which its own `peerDependencies` range - enforces. TypeScript 7's native (Go-based) compiler ships no public - compiler API at all — the classic `ts.Node`/`ts.forEachChild`/ - `ts.createProgram` surface this plugin is built on is gone, and the - `tsserver` protocol that loads plugins has been replaced by LSP. That - affects every TypeScript language service plugin, not just this one; - TypeScript 7.1 is expected to introduce a new (and different) programmatic - API. The library itself is unaffected and is tested against TypeScript 7 in - CI — this is exactly why the plugin lives in a separate package with a - separate version, so `@owlsql/core` isn't held to the plugin's narrower - range. - -## API reference - -| Export | Kind | Description | -| ------ | ---- | ----------- | -| `createTypedDb(executor)` / `createTypedDb(executor, options?)` | function | Build a schema-bound client. When passing `options`, `DB` and `Options` must **both** be given explicitly — `createTypedDb(executor, options)` with a single type argument is a compile error, not a silent no-op. `options.strict` enables [strict mode](#8-strict-mode--turn-typos-into-type-errors); `options.placeholders` enables [placeholder-style checking](#10-typed-parameters). | -| `TypedDb` | interface | The client; has `query(sql, ...params)`. | -| `TypedDbOptions` | interface | `{ strict?: boolean; placeholders?: PlaceholderStyle }`. | -| `Executor` | type | `(sql: string, params: readonly unknown[]) => Promise`. | -| `ExecutorResult` | type | `unknown[]` or `{ rows: unknown[]; meta?: QueryMeta }`. | -| `QueryMeta` | interface | `{ rowCount?; lastInsertRowid? }`, surfaced on the Ok result. | -| `PlaceholderStyle` | type | `'dollar' \| 'question' \| 'at'`. | -| `DialectExecutor