feat(translate): translation memory hints for changed units - #2296
Merged
Merged
Conversation
…rom the seed
A unit the seed does not cover but whose earlier wording the per-file seed
memory still holds (an edited sentence) is sent to the model together with
that previous source, its existing translation and the word-level changes
between the two versions, with the instruction to apply exactly these
changes. Measured on ru->en point edits: the median number of words changed
in the translation beyond the source edit goes from 9 to 0, the judge's
consistency score from 74 to 99, at about 2% more tokens per request. See
docs/specs/2026-09-22-translate-memory-hints-design.md.
- The per-file seed memory stores the source text instead of its hash
(seed file version 3); TranslationStore.hints() traces every unresolved
unit to the closest unused entry of the file (Dice over word bags, at
least 0.6, each entry used once).
- New utils/diff.ts: word-level changes and similarity; tags count as one
token so a placeholder does not outweigh the words around it.
- buildMessages() renders a memory block before the fragments ({{memory}}
placeholder for custom prompts); hints travel with fragments through
every retry.
- --no-memory-hints / memoryHints: false turns it off; the stat line
reports memory-hints: N and the run report cache.hints.
martyanovandrey
requested review from
a team and
diplodoc-bot
as code owners
September 22, 2026 16:57
martyanovandrey
requested review from
goldserg and
reazy015
and removed request for
a team
September 22, 2026 16:57
…n the memory block, one pass over the file memory - A negatable flag carries its default in args, so `memoryHints: false` in the config never reached the provider; the config is now consulted unless --no-memory-hints was given. - Seeds keep units in their XLIFF wrapper while fragments go out without it: the previous source and translation are unwrapped in the memory block, so the request reads the same way it was measured. - resolve() and hints() shared no work and the similarity re-split every pair into words: lookup() serves translations and hints in one pass, word bags are built once per text and pairs whose sizes cannot reach the threshold are skipped. A file of 1500 changed units takes ~0.2s instead of ~3s.
With --no-memory-hints (or memoryHints: false) the provider resolved the units through lookup() and threw the hints away. resolve() now takes the sequence match only, and lookup() runs only when hints are sent.
reazy015
approved these changes
Sep 23, 2026
| if (hinted[index]) { | ||
| stat.memoryHints++; | ||
| } | ||
| bufferTokens += tokens; |
Contributor
There was a problem hiding this comment.
Batches are sized by estimateTokens(text) per unit, but hints are not counted. renderMemory adds the full previous source and translation per hinted fragment, so a batch where every unit is hinted can exceed maxBatchTokens about threefold.
Beyond the tuned batch size the model tends to drift or drop fragments, so the very consistency this PR buys may suffer on long pages.
Should the hint's estimated tokens (source, translation, changes) be added to bufferTokens when buffering a unit, so maxBatchTokens keeps meaning the request size?
A hinted unit goes out with its previous source, translation and changes, about three times its own size, while batches were cut by the unit alone, so a batch of edited units could exceed maxBatchTokens threefold. The rendered memory entry now counts towards the batch; the oversize check stays on the unit, so a unit that fits alone is still sent with its memory.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
yfm translatewith a seed (yfm translate seed) reuses the existing translation of every unchanged unit and sends only the changed units to the model. Until now a changed unit arrived at the model alone, so a one-word edit in the source came back as a rewritten sentence: the reviewer of the translated page had to reread the whole paragraph to find the one real change, and the wording drifted between edits.Now a unit whose earlier wording the per-file seed memory still holds is sent together with that previous source, its existing translation and the word-level changes between the two versions, with the instruction to apply exactly these changes to the existing translation.
Design and measurements:
docs/specs/2026-09-22-translate-memory-hints-design.md.Why this shape
Measured on 37 changed units of the Tracker documentation (10 real point edits from the repository history, 27 synthetic),
deepseek-v4-flashat temperature 0 with the production prompt,glmas the judge; control run with the models swapped.The previous translation alone makes the model keep it even where the source changed (a removed anchor, a removed sentence); listing the changes fixes that. Other context (neighbours, the page, heading paths) was measured and rejected. Cost: about 250 tokens per request.
How
TranslationStore.hints()traces every unitresolve()leaves without a translation to the closest unused entry of the file memory: Dice coefficient over word bags, at least 0.6, every entry used once in document order.utils/diff.ts:wordChanges()(LCS over words, runs cut at 12 words) andsimilarity(). A tag counts as one token, so an XLIFF placeholder does not outweigh the words around it.buildMessages()renders a memory block right before the fragments; a custom prompt can place{{memory}}itself. Without hints the user message is unchanged.translateBatch()and every retry, so the untranslated retry resends the same prompt.--no-memory-hints(configmemoryHints: false) turns it off. The stat line printsmemory-hints: N, the run report exposescache.hints.Checked live with the built CLI on two Tracker pages seeded from their previous revision: both point edits went out with their memory (
memory-hints: 2), the heading and the edited list line kept their existing wording.