Skip to content

index @expr is unreachable: @fields is required on both index.lookup and identity.secondary, and @fields+@expr silently drops the fields #342

Description

@dmealing

Summary

@expr is registered on both index.lookup and identity.secondary, the migrate engine
builds it, and the Postgres emitter renders it — but the loader's @fields requirement
makes it unreachable as documented. The registry describes @expr as:

Raw key EXPRESSION for a functional/expression index (e.g. lower(email)). Used
INSTEAD of @fields — the index key is the expression rather than plain columns.

Yet omitting @fields fails to load on both node types, and the one spelling that does
load (@fields and @expr) silently discards the declared fields.

Net effect: an expression index cannot be expressed in metadata. An adopter has to keep it
as hand-written SQL outside the migration ledger, which then shows up as permanent
verify --db drift.

Version: 0.24.0 (npm @metaobjectsdev/cli@0.24.0; same result via the Python package).

Reproduction

Entity with a jsonb column, targeting Postgres:

{ "object.entity": { "name": "Notification", "children": [
  { "source.rdb": { "@table": "notifications" } },
  { "field.string": { "name": "id", "@required": true } },
  { "field.timestamp": { "name": "delivered_at" } },
  { "field.string": { "name": "payload", "@dbColumnType": "jsonb", "@required": true } },
  { "identity.primary": { "name": "pk", "@fields": ["id"] } },

  { "index.lookup": { "name": "pending_device_idx",
                      "@expr": "(payload->>'device_id')",
                      "@where": "delivered_at IS NULL" } }
] } }

meta verify

failed to load metadata: index.lookup "pending_device_idx" on "Notification"
has no @fields; at least one field is required

Matrix

Node attrs Result
index.lookup @expr only has no @fields; at least one field is required
identity.secondary @expr only is missing required attribute '@fields'
index.lookup @fields + @expr ✅ loads — but @fields is silently dropped

The third row is the concerning one. With @fields: ["delivered_at"] and
@expr: "(payload->>'device_id')", the emitted schema snapshot
(.metaobjects/migrations/.schema.postgres.json) records:

{ "columns": [], "expr": "(payload->>'device_id')", "name": "t2", "unique": false }

columns: [] — the declared field is gone, with no error and no warning. So the only
spelling that loads is one where part of the declaration is quietly ignored.

Why this looks like a validation gap rather than a missing feature

The capability is already implemented downstream of the loader:

  • migrate-ts expected-schema.js"An expression index keys off @expr (not
    @fields); a plain index needs @fields"
    , and it sets index.expr for both
    index.lookup and identity.secondary.
  • migrate-ts emit/postgres.js — renders CREATE [UNIQUE] INDEX … (keys)[ WHERE …],
    honoring @using / @where.
  • Introspection reads expression keys back (pg_get_expr), so drift comparison is
    already expression-aware.
  • The SQLite/D1 introspectors have explicit expression-index handling too, and a past fix
    restored @expr/@where/@orders there — so expression indexes are clearly intended
    to work end to end.

expected-schema.js already branches on "expression index vs plain index", so the
requirement that a plain index needs @fields appears to have been applied unconditionally
at the loader instead of being waived when @expr is present.

Suggested fix

Treat @fields as required unless @expr is present, on both index.lookup and
identity.secondary — matching what expected-schema.js already does. Then either:

  • reject @fields + @expr together as a contradiction (fail closed, since one is
    currently discarded), or
  • define and document a precedence rule.

Failing closed seems more consistent with the sealed-strict-registry posture: today the
combination is accepted and half-honored, which is the silent-wrong-output case the strict
loader generally exists to prevent.

Docs

If the @fields-always requirement is intentional, then the @expr description ("Used
INSTEAD of @fields") is wrong on both node types and should say what the supported
spelling actually is.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions