feat: implement queueing and reservation functionality - #405
Merged
Merged
Conversation
- Added InMemoryQueueStore class for managing job queues and reservations. - Introduced QueueStore module with required methods for enqueueing, reserving, releasing, and expiring reservations. - Created Reservation class to handle lease tokens and job metadata. - Updated job model documentation to include queueing and reservation contract details. - Added tests for InMemoryQueueStore and QueueStore to ensure correct behavior and error handling.
There was a problem hiding this comment.
Pull request overview
Adds the first runtime queueing/reservation layer to Karya, including an in-memory reference store, a backend contract, and supporting documentation/tests, to satisfy Issue #13’s “queueing and reservation behavior” milestone.
Changes:
- Introduces
Karya::QueueStorecontract andKarya::InMemoryQueueStoreimplementation for enqueue/reserve/release/expiration. - Adds
Karya::Reservationimmutable value object for reservation lease metadata. - Updates runtime job model docs to describe the queueing/reservation contract and adds specs for the new components.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/pages/runtime/job-model.md | Documents the queueing + reservation contract and lifecycle expectations. |
| core/karya/lib/karya/reservation.rb | Adds immutable Reservation value object with normalization/validation. |
| core/karya/lib/karya/queue_store.rb | Defines QueueStore interface and related error types. |
| core/karya/lib/karya/in_memory_queue_store.rb | Implements in-memory enqueue/reserve/release/expire behavior with mutex protection. |
| core/karya/lib/karya.rb | Wires new runtime components into the gem entrypoint. |
| core/karya/spec/karya/reservation_spec.rb | Tests reservation normalization/validation and expiration predicate. |
| core/karya/spec/karya/queue_store_spec.rb | Tests QueueStore interface enforcement (NotImplementedError). |
| core/karya/spec/karya/in_memory_queue_store_spec.rb | Exercises enqueue/reserve/release/expire flows and important edge cases. |
- Added validation for the `expired_tombstone_limit` parameter in the initializer to ensure it is a finite non-negative Integer. - Enhanced error handling by updating the message for duplicate reservation tokens to clarify that the token can be either active or expired. - Introduced new tests to cover cases for invalid `expired_tombstone_limit` values, ensuring robustness in input handling.
- Updated enqueue, reserve, and release methods to use specific error classes for better clarity. - Introduced InvalidQueueStoreOperationError for general queue operation errors. - Refined existing error handling to ensure invalid inputs are caught and reported accurately. - Added tests to verify that invalid inputs do not expire reservations and are handled correctly.
- Added checks to raise appropriate errors when a reservation is not found or expired. - Updated error messages for clarity, ensuring they accurately reflect the issue. - Enhanced test coverage to validate handling of blank and nil worker IDs during reservation.
- Clarified that automated linter feedback should not be a focus during reviews, as it is handled in CI. - Added emphasis on reviewing code clarity, logic simplification, error message quality, and test coverage completeness. - Included new review priorities for ambiguous error messages, duplicate logic, and incomplete test coverage.
- Added a new gemfile for tracking the Rails main branch to facilitate early regression detection. - Updated the gemfile.lock to include dependencies for the Rails main branch. - Modified the Gemfile.lock in `karya-roda` and `karya-sinatra` to upgrade `bigdecimal`, `json`, and `parser` versions for compatibility and performance improvements. - Enhanced the `bundle-update-all` and `ci-install-bundles` scripts to support multiple gemfiles in suites, ensuring that all relevant gemfiles are updated and installed. - Improved the `run-common` script to check for Rails versions and handle gemfile-specific commands, allowing for better integration with Rails applications. - Updated various run scripts (`run-reek-all`, `run-rspec-unit-all`, `run-rubocop-all`) to support running commands for multiple gemfiles, enhancing the testing and linting processes across different configurations.
- Introduced a new LeaseDuration class to validate and normalize lease durations. - Ensured that lease durations must be positive numbers and support various numeric types. - Updated the reserve method to utilize the new LeaseDuration class for improved error handling. - Added a test case to reject unsupported numeric lease durations, enhancing robustness.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes: #13