feat(ocr): entity chips under the recognized text - #150
Merged
Merged
Conversation
Recognized image text now shows what it can do. Under the lines, a chip row offers: - Open <host> for each http(s) link found anywhere in the text (bare domains the detector recognizes get their scheme), up to three; - Send email for mailto: addresses; - Translate when the text reads in a language other than the interface language AND an on-device engine can run now — an installed native pair or the language model. The translation renders beside the original, at the same body size, with Copy and Paste. Chips never appear for a recognized secret: a flagged text is not sent anywhere, not even to the on-device model. Only http, https and mailto ever open, re-checked at the call site because OCR output is untrusted. Chips wrap into rows instead of truncating, at 11 pt medium — the macOS floor for an interactive label, one step above the metadata around them. Tests: the detector (dedup, cap, unsafe schemes, mailto with a query), the translation offer with injected engines, and the OCR UI test asserts the link chip by host and that clicking it never leaves the app. The UI fixture gains a link line.
…ow layout Review of the entity chips found ten defects; this lands the fixes. - Copy translation and Paste translation now go through the same revalidation as every other copy from the section (private mode, deletion, expiry), and Translate itself revalidates before the text reaches the model. Copy translation confirms with the section's "Copied to clipboard" status. - The translation lifecycle is one enum (none, offered, running, done, failed): a finished translation keeps its target and survives an offer refresh; the task is keyed to the request and cancelled on a new request or when the section disappears, so a stale result can neither re-enable the chip nor land. - The translation block is capped like the peek's other long outputs (scrolling, 140 pt), so it never pushes the action list off the panel. - The email chip shows its address; two recipients can no longer read as the same chip, to the eye or to VoiceOver. - One FlowLayout in GanchoDesign replaces the private copies in both Intelligence screens and the chip-only ChipFlow. It measures and places against the same row width and offers each subview the row width, so a long host wraps inside its chip instead of running past the panel edge as a trusted-looking prefix. - The Translate offer asks SmartPasteService.route, the same decision on the same sanitized, clipped text translate sends, instead of re-deriving the routing matrix on the raw text. - openRecognizedEntity takes an ImageTextEntity: the detector is the one allowlist, and a launch the system refuses shows a toast. - LanguageName in GanchoDesign replaces three private copies of the language-name lookup. The paste-sink flag's doc names its side effects. - Tests: the offer is asserted to read the sanitized text; the UI test counts link chips instead of probing a fixed index.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Entity chips under the text recognized from an image, in the panel's peek (follow-up to #140).
http,httpsandmailtoever open, re-checked at the call site.How it was verified
make lint: 0 violations.make test: 1,143 tests in 170 suites.file:andjavascript:schemes failsunsafeSchemesAreDropped.ManualOCRUITests(3 tests) green locally; the link chip is asserted by host and clicking it never leaves the app under the paste sink. Hosted UI run dispatched on this head.Out of scope
iOS parity, and chips for phone numbers or dates (copying the line covers them).