Skip to content

feat(workflow): freeze a pinned version as the public copy - #7853

Draft
yangzhang75 wants to merge 1 commit into
apache:mainfrom
yangzhang75:pin/2-service
Draft

feat(workflow): freeze a pinned version as the public copy#7853
yangzhang75 wants to merge 1 commit into
apache:mainfrom
yangzhang75:pin/2-service

Conversation

@yangzhang75

@yangzhang75 yangzhang75 commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

A public workflow follows the author's latest content, as publishing has always done: every
save reaches the Hub immediately. This adds the other state — the author pins the version
they have now, and what the public sees stops moving until they pin again.

  • WorkflowPublishService owns the two states and the moves between them: publish,
    pinLatest, unpin, unpublish, statusOf. is_public stays the on/off switch;
    published_content is the pin, NULL while following.
  • A pin freezes everything on public show — the graph, the title, the description and the
    view the workflow opens in — because a copy that froze only its graph would still advertise
    a title nobody published. The default view matters because a form's definition rides inside
    the content: serving the author's live preference over a frozen graph would open a form on a
    copy that has none.
  • Three endpoints:
Endpoint Does
POST /workflow/pin/{wid} freezes the author's current version as the public copy; called again, moves the pin forward
DELETE /workflow/pin/{wid} drops the pin, back to following the latest
GET /workflow/publish-status/{wid} published, pinned, and whether a pin is holding edits back
  • Publishing and unpublishing move through the same service, so unpublishing drops the pin
    rather than leaving a private workflow carrying one. Re-publishing starts in the following
    state: coming back should not silently put old public content on show again.

  • One writer for the publish columns. Three paths could write them by accident, because
    each read a whole row (or took a whole Workflow from the client) and wrote it all back:

    • a save carrying a stale is_public would take a pinned workflow private underneath its
      own frozen copy, which the CHECK constraint refuses outright — leaving an editor that can
      no longer save. It now writes only name, description and content.
    • a rename or description edit wrote every column from a row it had read a moment
      earlier, so anything landing in between was silently rewritten to what that read had seen:
      a publish undone, a pin dropped, or — worst — a workflow the author had just unpublished
      put back on public show under its frozen copy. Each now writes only its own column.
    • creating a workflow clears the publish columns, so a request body cannot seed a
      published copy of its own choosing.

    Together with /set-default-view, which already wrote only its column, every endpoint now
    writes what it owns and nothing else: outside creation, WorkflowPublishService is the only
    writer of is_public and the frozen copy.

Nothing on screen changes. No read path consults the pinned copy yet and there is no UI:
every workflow stays in the following state it is in today. The endpoints answer, and nothing
calls them.

Any related issues, documentation, discussions?

Closes #7938
Part of #7828. Design discussion: #7128. Schema: #7851.

How was this PR tested?

25 cases in WorkflowPublishSpec:

  • the state machine — follow → pin → re-pin → unpin → unpublish → re-publish, and that a
    re-publish does not resurrect the previous pin;
  • what a pin freezes — the name, the description and the default view alongside the content,
    and that each of them moving afterwards is reported as an unpublished change;
  • the guards — no write access, not published, and a workflow that does not exist;
  • that a create cannot inject publish columns, and that a save, a collaborator's save and a
    rename all leave the publish state alone;
  • that a publish or an unpublish landing while a rename is in flight survives it. These two
    drive the interleaving off the statement itself rather than off a thread, so the ordering is
    the same on every run.

hasUnpublishedChanges compares the two copies as parsed JSON rather than as strings — the
same graph can come back with its keys in another order, and reporting that as an edit is an
alarm the author cannot clear. One case covers it.

The narrowed write paths were checked by mutation: restoring is_public to the save statement
turns three cases red, one of them on the CHECK constraint itself, and restoring the
read-modify-write to the rename path turns the two interleaving cases red — one showing the
publish reverted, the other showing the unpublished workflow back on public show.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-5)

@github-actions github-actions Bot added engine ddl-change Changes to the TexeraDB DDL labels Aug 23, 2026
@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@codecov-commenter

codecov-commenter commented Aug 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.91549% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.02%. Comparing base (cd4fd5a) to head (3f61548).
⚠️ Report is 5 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...shboard/user/workflow/WorkflowPublishService.scala 81.81% 1 Missing and 9 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7853      +/-   ##
============================================
- Coverage     94.03%   94.02%   -0.01%     
- Complexity     4771     4780       +9     
============================================
  Files          1196     1197       +1     
  Lines         48757    48827      +70     
  Branches       5916     5934      +18     
============================================
+ Hits          45850    45912      +62     
  Misses         1458     1458              
- Partials       1449     1457       +8     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø) Carriedforward from cd4fd5a
agent-service 99.32% <ø> (ø) Carriedforward from cd4fd5a
amber 89.89% <85.91%> (-0.01%) ⬇️
computing-unit-managing-service 73.67% <ø> (ø) Carriedforward from cd4fd5a
config-service 87.12% <ø> (ø) Carriedforward from cd4fd5a
file-service 87.91% <ø> (ø) Carriedforward from cd4fd5a
frontend 96.81% <ø> (ø) Carriedforward from cd4fd5a
notebook-migration-service 79.31% <ø> (ø) Carriedforward from cd4fd5a
pyamber 98.47% <ø> (ø) Carriedforward from cd4fd5a
workflow-compiling-service 77.19% <ø> (ø) Carriedforward from cd4fd5a

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 8 worse · ⚪ 7 noise (<±5%) · 0 without baseline

Compared against main 1cbe857 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 455 0.278 20,549/31,428/31,428 us 🔴 +17.0% / 🔴 +96.7%
🔴 bs=100 sw=10 sl=64 947 0.578 103,408/137,471/137,471 us 🔴 +12.5% / 🔴 +25.8%
bs=1000 sw=10 sl=64 1,146 0.699 873,581/924,823/924,823 us ⚪ within ±5% / 🟢 +13.9%
Baseline details

Latest main 1cbe857 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 455 tuples/sec 503 tuples/sec 755.36 tuples/sec -9.5% -39.8%
bs=10 sw=10 sl=64 MB/s 0.278 MB/s 0.307 MB/s 0.461 MB/s -9.4% -39.7%
bs=10 sw=10 sl=64 p50 20,549 us 17,563 us 12,938 us +17.0% +58.8%
bs=10 sw=10 sl=64 p95 31,428 us 32,035 us 15,980 us -1.9% +96.7%
bs=10 sw=10 sl=64 p99 31,428 us 32,035 us 19,233 us -1.9% +63.4%
bs=100 sw=10 sl=64 throughput 947 tuples/sec 1,031 tuples/sec 976.3 tuples/sec -8.1% -3.0%
bs=100 sw=10 sl=64 MB/s 0.578 MB/s 0.629 MB/s 0.596 MB/s -8.1% -3.0%
bs=100 sw=10 sl=64 p50 103,408 us 94,541 us 102,340 us +9.4% +1.0%
bs=100 sw=10 sl=64 p95 137,471 us 122,234 us 109,262 us +12.5% +25.8%
bs=100 sw=10 sl=64 p99 137,471 us 122,234 us 118,827 us +12.5% +15.7%
bs=1000 sw=10 sl=64 throughput 1,146 tuples/sec 1,156 tuples/sec 1,006 tuples/sec -0.9% +13.9%
bs=1000 sw=10 sl=64 MB/s 0.699 MB/s 0.706 MB/s 0.614 MB/s -1.0% +13.8%
bs=1000 sw=10 sl=64 p50 873,581 us 860,324 us 999,855 us +1.5% -12.6%
bs=1000 sw=10 sl=64 p95 924,823 us 952,383 us 1,042,833 us -2.9% -11.3%
bs=1000 sw=10 sl=64 p99 924,823 us 952,383 us 1,070,722 us -2.9% -13.6%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,439.28,200,128000,455,0.278,20549.22,31428.44,31428.44
1,100,10,64,20,2111.67,2000,1280000,947,0.578,103408.43,137470.74,137470.74
2,1000,10,64,20,17457.05,20000,12800000,1146,0.699,873581.49,924822.85,924822.85

@yangzhang75
yangzhang75 force-pushed the pin/2-service branch 3 times, most recently from a3c9eee to a70c881 Compare September 3, 2026 21:09
@github-actions github-actions Bot removed the ddl-change Changes to the TexeraDB DDL label Sep 3, 2026
@yangzhang75
yangzhang75 marked this pull request as ready for review September 3, 2026 22:14
@yangzhang75
yangzhang75 marked this pull request as draft September 3, 2026 22:15
A public workflow follows the author's latest content, as publishing has
always done. This adds the other state: the author pins the version they
have now, and the public copy stops moving until they pin again.

`is_public` stays the on/off switch; `published_content` is the pin, NULL
while following. `WorkflowPublishService` owns the two states, and three
endpoints expose them: POST and DELETE `/workflow/pin/{wid}` to pin and
unpin, GET `/workflow/publish-status/{wid}` for what the author is shown.
Publishing and unpublishing move through the same service, so unpublishing
drops the pin rather than leaving a private workflow carrying one.

Two paths are narrowed so a pin can hold. A save wrote the whole row back,
so a publish landing while a save was in flight was silently rolled back,
and a request body could set the publish columns itself; saves now write
only name, description and content. Creating a workflow clears the publish
columns for the same reason.

Nothing reads the pinned copy yet: every workflow is in the following
state it is in today, and nothing on screen changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Freeze a pinned version as the public copy

2 participants