Skip to content

mdcode: fields with no declared datatype are published as STRING, silently #307

Description

@feixufeixu

Found while working on a change in the same area, not from a bug report.

A model field with no declared datatype is published as STRING for both
dataType and metadataType, with no warning.

src/libts/semantic/knowledge_catalog.ts, columnDataType /
columnMetadataType — both fall through to STRING for undefined.

This is the normal path rather than an edge case: 13 of the 14 Ossie fixtures
declare no datatype at all, and 28 of 28 schema fields across the goldens
are STRING/STRING
— including o_totalprice and the other decimal
columns in star_orders_customer.

Reproduce

Push any model whose fields omit datatype (e.g. star_orders_customer.yaml)
and read the emitted schema aspect:

{"name": "o_totalprice", "dataType": "STRING", "metadataType": "STRING"}

Expected: something that does not assert a type the model never stated.

Four things that seem to interact here

It cannot warn, structurally. schemaAspectData(entity) takes no
warnings parameter. The metric path handles the identical situation and does
warn — metricAspectData(metric, warnings) defaults to NUMERIC and pushes a
message.

OTHER exists for this. metadataType's enum has it, and this same file
uses it for Opaque. STRING asserts a type; OTHER admits the absence of one.

undefined and 'String' are indistinguishable in the output, so a
consumer cannot tell "the author said string" from "the author said nothing" —
which for a catalog seems like the distinction worth keeping.

Opaque disagrees with itself: columnDataType('Opaque')'STRING'
while columnMetadataType('Opaque')'OTHER'.


cc @libei

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions