Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The same seeded qwen3tts request comes back different on every server instance. Within one instance it is bitwise repeatable, across instances it is not, and greedy decoding differs too. The talker decode graph is captured from a torch.compile wrapper in max-autotune mode, which benchmarks kernel candidates in each process and keeps the fastest. Instances end up with different kernels, the rounding differs, and after a few dozen sampling steps the token sequence diverges. Nine instances on an H200 gave nine different outputs.
This captures the eager kernels instead (compile=False on the talker_decode config, which is what cosmos3 already does). Three fresh instances now give identical seeded and greedy output, and greedy matches the eager path.
The trade-off is speed. Decode is about 20 percent slower per request on an H200 (0.65 s instead of 0.53 s for a five second clip, 10.6 ms per codec frame instead of 8.8), and startup is about 30 s faster because inductor no longer runs for this walk. Batched throughput was not measured. Full pytest passes, and test_qwen3_tts_model now asserts the capture config.