feat(datafusion): support REST Catalog CREATE FUNCTION#499
Merged
JingsongLi merged 3 commits intoJul 11, 2026
Conversation
leaves12138
requested changes
Jul 11, 2026
leaves12138
left a comment
There was a problem hiding this comment.
I found one blocking issue around complex argument/return types. The existing targeted suites and formatting checks pass, but focused regression cases with ARRAY<BIGINT> reproduce failures before the function can be created or executed.
leaves12138
approved these changes
Jul 11, 2026
leaves12138
left a comment
There was a problem hiding this comment.
Re-reviewed b0dbc50. The shared recursive SQL type renderer now handles both CREATE FUNCTION validation placeholders and REST SQL function return casts. The added ARRAY regressions pass, and I also verified MAP and ROW argument signatures locally. The persistent CREATE FUNCTION suite, REST SQL function suite, and formatting check all pass. LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add persistent
CREATE FUNCTIONsupport for SQL scalar functions in Paimon REST Catalogs. The implementation uses the existing REST SQL function expansion path to validate a candidate definition before persisting it, so newly created functions follow the same execution semantics as catalog functions read from REST.Changes
RETURNexpressions.LANGUAGE SQLis optional with SQL as the default, and determinism is inferred from the planned expression instead of requiringIMMUTABLE; explicitIMMUTABLEremains accepted for compatibility.IF NOT EXISTSandOR ALTER, including comments and quoted identifiers, without rewriting function bodies.Testing
cargo test -p paimon --test rest_object_models_testcargo test -p paimon --test rest_api_test --test rest_catalog_testcargo test -p paimon-datafusion --lib persistent_rest_catalog_function_cargo clippy -p paimon -p paimon-datafusion --all-targets -- -D warningscargo fmt --all -- --checkandgit diff --checkNotes
Persistent function creation is implemented by REST Catalog; other catalog implementations return
Unsupportedby default. Aggregate, table, multi-return, non-SQL, Stable/Volatile, lambda/file, temporary, alter, and drop function forms remain out of scope.A full local
paimon-datafusion --librun passed 260 tests; seven existing table-scan tests require pre-populatedpartitioned_log_tablefixtures under/tmp/paimon-warehouse, which are not present in this environment.