Overworld Waypoint1.5B integration

[garv901]

Overview

This PR adds Waypoint 1.5 1B, an interactive video world model, to mstar. Given a seed clip and a sequence of user actions, Waypoint maintains an internal world and streams generated RGB frames as that world advances.

Model shape

Prime once:
seed clip -> VAE encoder -> DiT [seed world state; no client output]-> VAE decoder

Roll out N times:
action + world state -> DiT -> VAE decoder -> 4 RGB frames -> client

This maps to three mstar nodes and two graph walks:

• prime initializes the DiT world state and the streaming decoder.
• rollout generates one latent frame per iteration and streams the decoded frames immediately.
• Each DiT rollout performs four denoising passes followed by one pass that commits the new frame to the world state.

Added video_frame as a separate modality

The existing video modality represents a complete encoded video artifact, such as an H.264 MP4. The container is finalized after generation and returned as one playable result. Waypoint has a different output shape. It generates four frames per rollout step and must send them immediately while the world continues running.video_frame therefore represents an ordered batch of raw RGB24 frames.

Resource boundary

In mstar, a resource is state or execution support that the engine must coordinate around a forward pass through admission, planning, stable allocation, commit, and cleanup.

Waypoint’s ring KV cache meets that definition: it is the persistent world state, must survive CUDA graph capture, assigns isolated world slots to requests, and must be reset when a request ends. FlexAttention is also an engine resource planned over that cache. The encoder and decoder remain normal model submodules because they do not require shared capacity or engine-level admission.

KV configuration

The previous KVConfig described both common KV geometry and paged-cache behavior. Waypoint uses a fixed-size ring that overwrites old frames, so fields such as page size, maximum pages, and offload policy do not apply.

We split the configuration into:

• KVConfig: geometry shared by all KV caches.
• PagedKVConfig: the existing append-only paged storage policy.
• RingKVConfig: Waypoint’s fixed-horizon, layer-specific ring policy.

KVSpec selects the appropriate manager from the concrete config. Existing models continue to use the same paged behavior, now named explicitly, while invalid storage-specific options fail instead of being silently ignored.

Async Execution

• Current design would cause loop external inputs to land inside ready_signals each iteration, hence would block speculative execution as it would not fall in either ready_next_iter or spec_signals bucket which is needed for same node speculation. Added a third bucket carried_names to address this.
• The DiT node's completion can allow 2 nodes to execute after it -> either the TAEHV decode node for the Nth iteration or N+1th on the DiT node itself. Such fanout scenarios are currently not addressed in mstar. For the first cut, I have fused these nodes but I will explore the ideal approach to handle this within this PR.

Benchmarking numbers [FLASH backend]

360p

Batch size	Startup (s)	TTFF p50 (ms)	TTFF p95 (ms)	Gap p50 (ms)	Gap p95 (ms)	Aggregate fps	On-time chunk %	Realtime
1	154.15	49.14	49.14	12.68	14.96	297.57	100.00	yes
2	279.32	87.46	117.46	13.84	18.81	520.21	100.00	yes
4	400.44	146.53	217.29	19.54	30.01	705.03	100.00	yes
8	276.33	268.05	308.57	32.87	52.12	856.86	98.41	yes
16	388.42	386.95	640.00	62.69	105.41	887.55	57.14	no

720p

Batch size	Startup (s)	TTFF p50 (ms)	TTFF p95 (ms)	Gap p50 (ms)	Gap p95 (ms)	Aggregate fps	On-time chunk %	Realtime
1	197.22	125.19	125.19	20.61	41.25	156.24	100.00	yes
2	471.55	208.42	278.84	42.04	82.86	177.41	82.54	no
4	637.73	382.25	585.63	81.68	130.97	183.72	17.46	no
6	786.87	560.36	841.88	124.51	193.03	180.00	4.76	no

Gap p50 = median across streams; Gap p95 = worst stream's p95. On-time chunk % = worst stream's fraction of chunks within the 66.7 ms budget. Realtime = every stream sustained ≥1× with gap p95 ≤ 66.7 ms and no stalls. Headline: 360p realtime to batch 8; 720p realtime only at batch 1.

Follow-up TODOs

• Async sched enabled but current implementation fuses the DiT and Decoder nodes into 1 which is not ideal.
• Add long-lived interactive sessions. Requests currently provide all actions up front. Real interaction requires live action ingress, reconnect and cancellation behavior, idle timeouts, and a policy for late or missing actions.
• Finish PR cleanup. Resolve outstanding review comments, remove temporary implementation notes, and retain only NVTX markers with a clear long-term profiling use.

[stephen-dwq]

In the new KV-ring resource:

Writes can only accept a fixed number of tokens per step; however, I am not sure if future models with ring KV mechanics will depart from this, or if it will be easy to book-keep for a dynamic write-size system.

Why is admit restricted to max_batch_size=1, with the restriction not applying elsewhere? Why can't we admit multiple RID at once?

Nit: num_worlds (in kv/ring/) is waypoint-specific, and should be replaced with num_slots or num_sessions; world_idx should be replaced by span_idx.

[garv901]

In the new KV-ring resource:

Writes can only accept a fixed number of tokens per step; however, I am not sure if future models with ring KV mechanics will depart from this, or if it will be easy to book-keep for a dynamic write-size system.

Why is admit restricted to max_batch_size=1, with the restriction not applying elsewhere? Why can't we admit multiple RID at once?

Nit: num_worlds (in kv/ring/) is waypoint-specific, and should be replaced with num_slots or num_sessions; world_idx should be replaced by span_idx.

@stephen-dwq Yeah that makes although for the scope of this model port I was thinking we can stuck to the fixed KV ring structure ? I have enabled batching in my latest commit, had deferred for ease of implementation and code review for myself initially. Addressed the nit.