Validate object-form semantic model dataset sources - #5406
mattfaltyn wants to merge 2 commits into
Conversation
| semantic_model: | ||
| type: string | ||
| description: The Apache Ossie semantic model serialized as a JSON string. | ||
| description: The Apache Ossie semantic model serialized as a JSON array string. |
There was a problem hiding this comment.
Semantic model itself has changed to a single object instead of an array. The array format only exist in a short period of time before 0.2.0.
There was a problem hiding this comment.
Semantic model itself has changed to a single object instead of an array.
Great catch—thank you! I updated the implementation so the single-object form now goes through the same dataset-source validation, while retaining the array path for compatibility. I also restored the object-form OpenAPI example and added focused regression coverage for an unresolved source in an object document.
There was a problem hiding this comment.
I don't think we need to support the old version(0.1.x), given it is beta and created before the Apache Ossie incubation. Also it's a incompatible change, we may avoid that given Ossie community has moved on.
There was a problem hiding this comment.
I don't think we need to support the old version (0.1.x)
Agreed on not preserving 0.1.x just for compatibility. One detail: both 0.1.1 and the linked 0.2.0.dev0 schema define semantic_model as an array; only the outer document is an object. Did you mean Polaris should intentionally unwrap a single model per entity? If so, I can align the contract.
There was a problem hiding this comment.
Sorry, I thought it was changed to a single object from array already. Let me sync with the Ossie community a bit.
There was a problem hiding this comment.
Hi @mattfaltyn , we have fixed it in the Apache Ossie repo, Ossie PR #383.
mattfaltyn
changed the title
3 participants
Summary
Validate every
dataset.sourcein both single-object and array-form semantic-model documents.Previously,
resolveAndValidateSourcesreturned without checking any source when a client suppliedan object, so create and update could persist dangling table or view references.
The OpenAPI example remains in the single-object form. Array documents continue through the same
source validation for compatibility. Full Apache Ossie JSON Schema validation and document-size
limits remain tracked by #5205.
Fixes #5405.
Validation
./gradlew :polaris-extensions-semantic-models:test --tests 'org.apache.polaris.service.catalog.semanticmodel.SemanticModelCatalogTest'./gradlew format compileAll --max-workers=1./gradlew :polaris-extensions-semantic-models:check --max-workers=1All commands pass on the final commit with OpenJDK 21.
AI assistance
AI assistance was used to investigate the defect, prepare the focused change, and run validation.
The author reviewed and understands the implementation.
Checklist
CHANGELOG.md(if needed)site/content/in-dev/unreleased(not needed; the existing object-form OpenAPI example remains accurate)