Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,12 @@ import { DoSqliteDriver } from "typegres/drivers/do"; // Cloudflare Durable

const db = typegres();

// Pick the one you're running against — the driver names the backend, and
// `db` takes its dialect from it:
db.connect(PgDriver.create(process.env.DATABASE_URL!));
db.connect(await PgliteDriver.create()); // the one async driver: booting WASM is real I/O
db.connect(SqliteDriver.create("dev.db")); // omit the filename for :memory:
db.connect(DoSqliteDriver.create(ctx.storage)); // in the DO constructor — no npm peer needed
db.connect(await PgliteDriver.create()); // the one async driver: booting WASM is real I/O
```

Drivers are imported explicitly from `typegres/drivers/*` so optional peers
Expand All @@ -93,12 +95,14 @@ stay out of bundles that never use them — install only the one you need.
With exactly one connection (the Durable Object model), it's also the
default: `.execute()` / `.live()` take no argument, and you can ignore what
`connect` returns. Pass a `Connection` explicitly when you have several —
read replicas, database-per-tenant, or a transaction's `tx`.
read replicas, database-per-tenant, or a transaction's `tx`. Several is fine
as long as they agree on dialect; the schema classes compiled against one.

## How it works

1. **Types codegen'd from the Postgres/SQLite catalog/docs.** all base types, full
method/operator coverage, nullability tracked at the type level.
1. **Types codegen'd from the engine itself.** Postgres from its catalog,
SQLite from its docs — all base types, full method/operator coverage,
nullability tracked at the type level.
2. **Object-capability queries.** Clients can only reach what you've exposed
as `@expose` methods — columns, relations, scoped reads, mutations. The class
surface is the contract; the schema underneath is free to move.
Expand Down
106 changes: 74 additions & 32 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,34 +54,60 @@ place, in one language.

## Runtime architecture

### `Driver` vs `Database`

- `Driver` is the low-level connection layer (`PgDriver`, `PgliteDriver`).
It exposes `execute(sql)`, `runInSingleConnection(fn)`, `close()`.
- `Database` is the typed query surface on top — what user code interacts
with. It holds a `Driver` and a query/hydrate/transaction API.

### Single-class Database, two states

A `Database` is either **pool-backed** (every execute routes through the
### `Driver` vs `Database` vs `Connection`

- `Driver` is the low-level connection layer (`PgDriver`, `PgliteDriver`,
`SqliteDriver`, `DoSqliteDriver`). It exposes `execute(sql)`,
`runInSingleConnection(fn)`, `close()`, and a `dialect`. Each lives at its
own entry point (`typegres/drivers/*`) so a bundle only ever resolves the
optional peer it actually imports.
- `Database` is the schema handle: provenance identity and the `Table`
factory, no driver of its own. `typegres()` constructs one synchronously,
so table classes can be declared at module load without a top-level await.
- `Connection` is the runtime handle — a `Database` plus a `Driver`, with
the execute/hydrate/transaction/live API. `db.connect(driver)` mints one.

`connect` is synchronous for every driver but PGlite, whose `create()` boots
WASM and is awaited by the caller. Multiple `connect` calls are allowed (test
+ prod, worker pools, read replicas, database-per-tenant); they share the
schema provenance but talk to independent drivers.

The dialect belongs to the driver, not the schema. `db.dialect` is a
passthrough to the first driver ever connected — reading it before any
`connect` throws rather than defaulting, since the dialect gates SQL
rendering and builder-time checks. A later `connect` whose driver disagrees
is rejected: the schema classes compiled against one dialect.

With exactly one pool-backed connection (the Durable Object model),
`db.defaultConnection` makes it implicit — terminators like `.execute()` and
`.live()` take no argument. Zero or several attached is ambiguous and throws,
so a `Connection` must be passed explicitly.

### Single-class Connection, two states

A `Connection` is either **pool-backed** (every execute routes through the
driver's pool) or **transaction-bound** (carries a single-connection
`ExecuteFn`). Both are instances of the same class. `transaction(fn)` hands
the callback a transaction-bound `Database`:
`ExecuteFn` and no bus of its own). Both are instances of the same class.
`transaction(fn)` hands the callback a transaction-bound `Connection`:

```ts
await db.transaction(async (tx) => {
await conn.transaction(async (tx) => {
await tx.execute(User.insert(...));
await User.from().execute(tx); // fluent form
});
```

There is no `AsyncLocalStorage` threading ambient context — the `tx` is
passed explicitly. Nested calls flatten because `Transaction.transaction(fn) =
fn(this)`, so callees that accept a `Database` don't have to know whether
they're getting the pool or a txn.
passed explicitly. Nested calls flatten because `transaction(fn) = fn(this)`,
so callees that accept a `Connection` don't have to know whether they're
getting the pool or a txn.

Transactions use pg's default isolation. No stricter level is imposed by the
framework.
Transactions default to the session's ambient isolation. `transaction({
isolation }, fn)` picks a level explicitly (pg only — sqlite transactions are
serializable by nature). Since pg can't promote isolation after the first
query, a nested request stronger than the active level throws rather than
silently downgrading, and any explicit level nested inside an ambient txn
throws too — we can't prove what the outer one got.

### Query builders and terminators

Expand All @@ -96,9 +122,10 @@ type level.
scope minted by `bind()`. Aliases are ephemeral to compilation, never
stored on classes, so client code can't fabricate references to tables
or rows outside the scope it was handed.
- `.execute(db)`, `.hydrate(db)`, `.one(db)`, `.maybeOne(db)` are fluent
terminators that accept any `Database` (pool or tx); `db.execute(...)` /
`db.hydrate(...)` are the non-fluent equivalents.
- `.execute(conn)`, `.hydrate(conn)`, `.one(conn)`, `.maybeOne(conn)`,
`.live(conn)` are fluent terminators that accept any `Connection` (pool or
tx), or none at all to use `db.defaultConnection`; `conn.execute(...)` /
`conn.hydrate(...)` are the non-fluent equivalents.

`hydrate` materializes rows as class instances — each column field is an
`Any` wrapping a `CAST(param)` of the value, so methods on the class
Expand All @@ -107,35 +134,50 @@ without breaking the capability chain.

## Type system

All Postgres types are represented as TS classes. Functions are methods on
Each dialect's types are represented as TS classes. Functions are methods on
those classes. Nullability is tracked in the `N extends number` type
parameter (`0 = null`, `1 = non-null`, `0 | 1 = maybe null`).

Full hierarchy: `Any` → `Anycompatible` → `Anyelement` → `Anynonarray` →
concrete types. Generic container types (`Anyarray<T>`, `Anyrange<T>`) wire
through `.of()`.
The Postgres hierarchy: `Any` → `Anycompatible` → `Anyelement` →
`Anynonarray` → concrete types. Generic container types (`Anyarray<T>`,
`Anyrange<T>`) wire through `.of()`. SQLite has the same shape over its six
storage classes (`Any`, `Integer`, `Real`, `Text`, `Blob`, `Bool`).

## Codegen

Types under `src/types/generated/` are generated from the pg catalog
(`pg_type`, `pg_proc`, `pg_operator`) via pglite introspection:
Both dialects emit through the same emitter (`src/types/emission/`); they
differ only in where the facts come from:

- `src/types/postgres/generated/` — introspected from the pg catalog
(`pg_type`, `pg_proc`, `pg_operator`) via pglite.
- `src/types/sqlite/generated/` — derived from committed per-page facts
extracted from the SQLite docs, since SQLite has no catalog to query.
`signatures.verify.test.ts` checks every claim against the real engine and
gates on completeness: each `pragma_function_list` entry is either covered
by the facts or explicitly excluded.

```
npm run codegen
```

The generated files are committed. `npm run codegen:check` regenerates into
a temp dir and diffs against the committed copies — CI runs this to catch
drift between the pg version and the checked-in output.
The generated files are committed. `npm run codegen:check` regenerates both
trees into a temp dir and diffs against the committed copies — CI runs this
to catch drift between the engines and the checked-in output.

Table codegen is separate: `npx tg generate` introspects a user's schema and
writes typed Table files into their project (uses `typegres.config.ts`).

## Raw SQL

`sql` is the escape hatch — a tagged template returning an immutable `Sql`
builder. Supports `sql.param`, `sql.raw`, `sql.ident`, `sql.join`. Fragments
compose via template nesting. Compiles to pg (`$1`) or sqlite (`?`) style.
builder. Supports `sql.param`, `sql.raw`, `sql.join`. Fragments compose via
template nesting. Compiles to pg (`$1`) or sqlite (`?`) style.

Schema-referencing identifiers go through `db.scopedIdent(name)` rather than
a bare `sql.ident` helper: an `Ident` must carry its `Database` to survive the
compile-time provenance check. The `Ident` class is exported for
library-internal callers that construct untagged identifiers inline (CTE
aliases, output column labels).

## Development environment

Expand Down
3 changes: 2 additions & 1 deletion examples/sqlite/src/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ import { SqliteDriver } from "typegres/drivers/sqlite";

// Synchronous end to end — no top-level await: `typegres()` is a
// module-load-safe schema handle, and better-sqlite3 opens the database on
// construction. The tests use `:memory:` (the `sqlite()` default) so each
// construction. The tests use `:memory:` (`SqliteDriver.create()`'s default
// when no filename is given) so each
// vitest run is hermetic; the `tg generate` CLI reads schema from the
// `./dev.db` file produced by `npm run migrate`.
export const db = typegres();
Expand Down
14 changes: 10 additions & 4 deletions site/src/lib/share-utils.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,13 @@ describe("Share Utils", () => {
it("should handle medium code snippets", () => {
const code = `
import { typegres } from 'typegres';
import { PgliteDriver } from 'typegres/drivers/pglite';

const db = typegres();
const conn = db.connect(await PgliteDriver.create());

async function main() {
const db = await typegres({ type: 'pglite' });


const users = await db.sql\`
SELECT id, name, email
FROM users
Expand Down Expand Up @@ -122,11 +125,14 @@ const nested = \`Template \${literal} with \\\`backticks\\\`\`;
it("should keep URLs under 2000 characters for typical code", () => {
const typicalCode = `
import { typegres } from 'typegres';
import { PgliteDriver } from 'typegres/drivers/pglite';
import { Users, Posts, Comments } from './schema';

const db = typegres();
const conn = db.connect(await PgliteDriver.create());

async function main() {
const db = await typegres({ type: 'pglite' });


// Get users with their post count
const usersWithPosts = await db
.select('u.id', 'u.name', 'u.email')
Expand Down
4 changes: 2 additions & 2 deletions site/src/pages/_PlayActiveArea.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
// that needs the live PGlite-backed `client`. Splitting it out of
// _PlayPageInner means the shell (header / Monaco editor / file
// tree) can render before runtime.ts finishes its top-level
// `await typegres({ type: "pglite" })`. Suspense in the parent
// shows a "booting" placeholder during that ~1-2s wait.
// `await PgliteDriver.create()`. Suspense in the parent shows a
// "booting" placeholder during that ~1-2s wait.

import { useEffect, useMemo, useRef, useState } from "react";
import * as monaco from "monaco-editor";
Expand Down
8 changes: 4 additions & 4 deletions src/live/ISSUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ applies to both dialects.

1. **Public API gap, closed.** Opt-in is uniformly
`db.Table("notes", { live: true })`; the live engine is wired at
`db.attach(driver, busOpts?)` with no start/stop lifecycle — sqlite
capture is active from attach, the pg poller starts lazily on first
`db.connect(driver, busOpts?)` with no start/stop lifecycle — sqlite
capture is active from connect, the pg poller starts lazily on first
`.live()` use, and `close()` tears the engine down. The only pg
ceremony left is `ensurePgLiveEventsTable(conn)` for the events-table
DDL (a migration concern, deliberately explicit).
Expand Down Expand Up @@ -213,11 +213,11 @@ applies to both dialects.

14. **Multi-connection sqlite live / `LiveDriver` (parked).** Two
Connections attached to one sqlite driver share the statement clock
(`SyncDriver.liveSeq`) but NOT a bus — each attach() makes its
(`SyncDriver.liveSeq`) but NOT a bus — each connect() makes its
own, and events captured through one Connection never reach the
other's subscribers. Unexercised today (a DO is one driver, one
Connection). The named future shape: a user-constructed `LiveDriver`
wrapper (`db.attach(new LiveDriver(new SqliteDriver(...)))`) owning
wrapper (`db.connect(LiveDriver.create(SqliteDriver.create(...)))`) owning
the clock AND a per-driver bus, reverting the base drivers to dumb
pipes. Notes from the design discussion: a per-statement driver
callback cannot absorb capture (CompiledSql has no table/column/
Expand Down
Loading