Read the recent activity log without loading the whole file

[elkampu]

Goal

GetEventsAsync reads the complete JSONL activity log before taking the newest entries, so memory and latency grow with every long-running session. Read the tail with working memory independent of total file size.

Acceptance criteria

• Working memory is bounded by a fixed read buffer plus the bytes needed for the current line and the N returned records.
• Newest means physical JSONL file order, not CaptureEvent.At; malformed records do not consume the requested valid-event limit.
• Include a syntactically valid final JSON record even without a trailing newline. Skip a malformed or truncated tail and continue to older valid records.
• Handle CRLF and LF delimiters and multibyte UTF-8 code points split across a buffer boundary.
• Open the log with FileShare.ReadWrite, snapshot its starting length, and ignore an append that is incomplete or occurs after that snapshot.
• Pass request cancellation into the internal read path and use HttpContext.RequestAborted from the endpoint.
• Preserve the existing 1–500 limit, newest-first response order, on-disk JSONL format, and public endpoint shape.
• FrameCatalogServiceTests covers limits 1 and 500, a large file, a valid unterminated final record, a malformed tail, and CRLF/UTF-8 data crossing buffer boundaries.

Please comment with your proposed algorithm and edge-case strategy before starting. A draft PR is welcome.

[iibaranov-IG]

Proposed algorithm and edge-case strategy:

• open the JSONL file read-only with FileShare.ReadWrite and snapshot Length once
• walk backward from that length in a fixed-size byte buffer, splitting on LF bytes (safe for UTF-8 because continuation bytes cannot equal 0x0A) and trimming an optional CR
• retain only the current reversed line plus serialized bytes for at most N returned records; decode each complete candidate once in forward byte order
• treat a valid unterminated final record as a record, skip malformed/truncated candidates without consuming the valid-event limit, and preserve physical newest-first order independent of event timestamps
• check cancellation between reads and candidate parses, and pass HttpContext.RequestAborted from the endpoint
• add tests for limits 1/500, a large file, valid and malformed unterminated tails, CRLF, and a multibyte code point split at the buffer boundary.

The on-disk format and response contract remain unchanged.