Skip to content

Update magnolify to 0.9.8; adopt portable Timestamp encoding for IcebergIO - #6000

Merged
shnapz merged 2 commits into
mainfrom
akabas/upgrade-magnolify-9-8
Sep 17, 2026
Merged

shnapz merged 2 commits into
mainfrom
akabas/upgrade-magnolify-9-8

Conversation

@shnapz

@shnapz shnapz commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator
  • Bump magnolify from 0.9.7 to 0.9.8
  • Switch IcebergIOIT from logical.millis._ (joda-backed DATETIME) to logical.timestamp.micros._ (Beam's portable Timestamp logical type at microsecond precision), which is what IcebergIO requires as of Beam 2.76.0
  • Add LocalDateTime / unzoned timestamp coverage to IcebergIOIT (from Add unzoned timestamp type test to IcebergIOIT #5999)

Context

Beam 2.76.0 changed IcebergIO's timestamptz mapping from DATETIME to Timestamp.MICROS (apache/beam#39344). With the old millis._ import, reads now fail with ClassCastException: java.time.Instant cannot be cast to org.joda.time.Instant.

Magnolify 0.9.8 (spotify/magnolify#1392) adds logical.timestamp.{millis,micros,nanos} which map Instant to Beam's portable Timestamp at the declared precision. The old encodings remain available under logical.compat.*; the bare logical.millis/micros/nanos objects are deprecated aliases that preserve 0.9.7 behavior.

Verified locally

All three scenarios tested against a real Iceberg REST catalog container:

Import Compat flag Result
timestamp.micros._ none pass
millis._ none failClassCastException on read
millis._ --updateCompatibilityVersion=2.75.0 pass

Test plan

  • CI green (build + unit tests across 2.13/2.12)
  • IcebergIOIT passes (already verified locally)

clairemcginty and others added 2 commits September 17, 2026 15:53
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.99%. Comparing base (1d4bc82) to head (86949e0).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6000   +/-   ##
=======================================
  Coverage   61.98%   61.99%           
=======================================
  Files         323      323           
  Lines       11743    11743           
  Branches      839      839           
=======================================
+ Hits         7279     7280    +1     
+ Misses       4464     4463    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@shnapz
shnapz merged commit 719c78d into main Sep 17, 2026
11 checks passed
@shnapz
shnapz deleted the akabas/upgrade-magnolify-9-8 branch September 17, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants