Skip to content

Add stream reading guidance: ReadStreamToEnd and memory semantics - #2

Open
alexeyzimarev wants to merge 1 commit into
mainfrom
update/streaming-reads
Open

Add stream reading guidance: ReadStreamToEnd and memory semantics#2
alexeyzimarev wants to merge 1 commit into
mainfrom
update/streaming-reads

Conversation

@alexeyzimarev

Copy link
Copy Markdown
Contributor

Follows Eventuous/eventuous#568 (fixes Eventuous/eventuous#567), which made KurrentDB reads truly streaming and added the ReadStreamToEnd extension.

  • skills/eventuous-dotnet: new "Reading Event Streams" section — read semantics per store, ReadStreamToEnd example, pageSize/failIfNotFound options, ReadStream as the array variant
  • skills/eventuous-dotnet-kurrentdb: note that KurrentDBEventStore reads hold at most one deserialized event at a time, pointer to ReadStreamToEnd
  • agents/eventuous-expert: new opinionated default — read whole streams with ReadStreamToEnd, never ReadEvents with int.MaxValue

Left the plugin version in .claude-plugin/plugin.json untouched — bump it if this should ship as a release.

🤖 Generated with Claude Code

Document the IEventReader read semantics (KurrentDB streams events as they
arrive, relational stores buffer up to count) and steer agents to
ReadStreamToEnd for whole-stream reads instead of ReadEvents with
int.MaxValue. Matches Eventuous/eventuous#568.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Document streaming read semantics and recommend ReadStreamToEnd for whole streams

📝 Documentation ✨ Enhancement 🕐 10-20 Minutes

Grey Divider

AI Description

• Document per-store IEventReader read/memory semantics (streaming vs buffered pages).
• Add ReadStreamToEnd guidance and example; discourage ReadEvents(..., int.MaxValue).
• Update KurrentDB skill and expert agent defaults to prefer paged, bounded-memory reads.
Diagram

graph TD
  A["Developer/Agent"] --> B["eventuous-expert.md"] --> C["Guidance: ReadStreamToEnd"] --> D["Eventuous IEventReader"]
  A --> E["eventuous-dotnet SKILL"] --> C
  A --> F["KurrentDB SKILL"] --> G["KurrentDB read semantics"] --> C

  subgraph Legend
    direction LR
    _actor["Doc consumer"] ~~~ _doc["Docs/Agent rules"] ~~~ _api["Library API"]
  end
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Keep `ReadEvents` but standardize on a bounded count
  • ➕ No need to teach an additional helper/extension method
  • ➕ Makes buffering behavior explicit at the call site (count chosen intentionally)
  • ➖ Easy for callers/agents to regress to int.MaxValue or other unbounded patterns
  • ➖ Doesn’t communicate the “read to end” intent as clearly as a dedicated API
2. Add a Roslyn analyzer or lint rule to forbid `ReadEvents(..., int.MaxValue)`
  • ➕ Prevents the problematic pattern mechanically across codebases
  • ➕ Scales better than relying on documentation/agent guidance
  • ➖ More implementation and maintenance effort; requires distribution/installation
  • ➖ Out of scope for a documentation-focused PR
3. Provide a cookbook sample project demonstrating bounded paging patterns
  • ➕ Gives copy/paste-ready reference code beyond a short snippet
  • ➕ Can show store-specific tuning (pageSize, not-found behavior) in context
  • ➖ Higher documentation surface area to keep current
  • ➖ More effort than updating the existing skills/agent guidance

Recommendation: The PR’s approach (explicitly recommending ReadStreamToEnd and documenting store-specific memory behavior) is the best fit for quickly correcting common misuse and aligning guidance with the updated Eventuous behavior. If misuse continues in practice, consider a follow-up analyzer to enforce the rule mechanically.

Files changed (3) +19 / -0

Documentation (3) +19 / -0
eventuous-expert.mdSet agent default to use ReadStreamToEnd for whole-stream reads +1/-0

Set agent default to use ReadStreamToEnd for whole-stream reads

• Adds an opinionated guideline to read full streams via 'IEventReader.ReadStreamToEnd()' and explicitly discourages 'ReadEvents' with 'int.MaxValue' to avoid unbounded reads.

agents/eventuous-expert.md

SKILL.mdClarify KurrentDB streaming memory semantics and point to ReadStreamToEnd +2/-0

Clarify KurrentDB streaming memory semantics and point to ReadStreamToEnd

• Documents that KurrentDB reads stream events as they arrive and hold at most one deserialized event at a time, independent of requested count. Recommends 'ReadStreamToEnd' over 'ReadEvents(..., int.MaxValue)' for whole-stream reads.

skills/eventuous-dotnet-kurrentdb/SKILL.md

SKILL.mdAdd “Reading Event Streams” section with ReadStreamToEnd example and options +16/-0

Add “Reading Event Streams” section with ReadStreamToEnd example and options

• Introduces a new section describing 'IEventReader' read semantics across stores (KurrentDB streaming vs relational buffering). Provides a 'ReadStreamToEnd' example and notes 'pageSize', 'failIfNotFound', and the array-returning 'ReadStream' alternative.

skills/eventuous-dotnet/SKILL.md

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. ReadStream memory cost unclear 🐞 Bug ⚙ Maintainability
Description
The new docs say ReadStream "does the same paged read" but don’t explicitly warn that returning
StreamEvent[] necessarily materializes the entire stream in memory (memory grows with stream
length, regardless of page size). This can lead users to inadvertently load very large streams into
memory when they only needed streaming iteration.
Code

skills/eventuous-dotnet/SKILL.md[265]

+Options: `pageSize` tunes the page size; `failIfNotFound: false` yields nothing instead of throwing `StreamNotFound`. The `ReadStream` extension method does the same paged read and returns `StreamEvent[]` if you need the whole stream as an array.
Evidence
The section emphasizes bounded-memory streaming via ReadStreamToEnd, then introduces ReadStream
as a paged read that returns an array, but does not explicitly call out that an array implies full
materialization and memory proportional to total stream size.

skills/eventuous-dotnet/SKILL.md[255-266]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The docs mention that `ReadStream` performs a paged read and returns `StreamEvent[]`, but they don’t clearly state that the returned array requires holding the whole stream in memory (page size only bounds the fetch buffer, not the final result).

### Issue Context
This is in the newly added "Reading Event Streams" section, immediately after recommending `ReadStreamToEnd` for bounded-memory streaming.

### Fix Focus Areas
- skills/eventuous-dotnet/SKILL.md[253-266]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can enable the Remediation agent and Qodo fixes findings in a dedicated fix PR

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

}
```

Options: `pageSize` tunes the page size; `failIfNotFound: false` yields nothing instead of throwing `StreamNotFound`. The `ReadStream` extension method does the same paged read and returns `StreamEvent[]` if you need the whole stream as an array.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. Readstream memory cost unclear 🐞 Bug ⚙ Maintainability

The new docs say ReadStream "does the same paged read" but don’t explicitly warn that returning
StreamEvent[] necessarily materializes the entire stream in memory (memory grows with stream
length, regardless of page size). This can lead users to inadvertently load very large streams into
memory when they only needed streaming iteration.
Agent Prompt
### Issue description
The docs mention that `ReadStream` performs a paged read and returns `StreamEvent[]`, but they don’t clearly state that the returned array requires holding the whole stream in memory (page size only bounds the fetch buffer, not the final result).

### Issue Context
This is in the newly added "Reading Event Streams" section, immediately after recommending `ReadStreamToEnd` for bounded-memory streaming.

### Fix Focus Areas
- skills/eventuous-dotnet/SKILL.md[253-266]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KurrentDB ReadEvents buffers the entire requested range before yielding — IAsyncEnumerable is not actually streaming

1 participant