fix(queue): make queue:work actually process jobs, and say what it did - #8
Merged
Conversation
A job class only reaches the JobRegistry when it is constructed, and a dedicated `queue:work` process constructs none of the application's jobs. Its registry is therefore empty, every payload it pops throws "Job X is not registered", and — because the failure path resolves the payload again to reach the `failed` hook — the job is deleted after exhausting its tries with nothing recorded anywhere. Add `loadJobs()`, which imports the job modules under `src/app/jobs` (or the build output) and registers every concrete class it finds. Detection tests for `handle` on the prototype rather than `instanceof Job`: a module loaded through jiti carries its own copy of the base class, so an identity check would reject every job it found. An abstract base declares no `handle` of its own and is skipped. Back the registry with a global symbol for the same reason, so a job module evaluated more than once registers into the one map the worker resolves against.
A worker swallowed every job failure: nothing was logged, `--once` reported "Processed one job." whether or not the job succeeded, and the daemon printed nothing at all. A queue whose jobs all fail was therefore indistinguishable from an idle one. - Worker gains `on()` handlers (onProcessing/onProcessed/onFailed), and `queue:work` prints each outcome — including whether a failed job is going back for another attempt or has failed permanently. - `--stop-when-empty` and `--max-jobs` never took effect: musket hands over commander's parsed options, which are camelCased, so the kebab-case keys always read as undefined. Both spellings are accepted now. - The banner claimed `[default] connection` regardless of which connection was resolved, and never named the queue being worked. - Load the application's job classes before working, so a payload can be turned back into a job. - Boot the ORM for `--once` too, not only for the daemon.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A dedicated
queue:workprocess could not run most of an application's jobs, and gave no sign of it. Found while investigating an app whose Redis queues had grown to 950 jobs, every one of them still atattempts: 0.Four defects compounded into "the queue is never processed":
JobRegistryis populated in theJobconstructor, and a worker process constructs none of the app's jobs — so its registry is empty and every payload it pops throwsJob "X" is not registered. Worse,handleFailureresolves the payload again to reach thefailedhook, which throws too, so the job is deleted after its tries with nothing recorded.Worker.processswallowed the error, the daemon printed nothing per job, and--oncereportedProcessed one job.even when the job failed.--stop-when-emptyand--max-jobsdid nothing. Musket hands over commander's parsed options, which are camelCased, sothis.option('stop-when-empty')always readundefined. A drained queue kept the daemon spinning.defaultregardless of the resolved connection, and never named the queue.What changed
@arkstack/jobsgainsloadJobs(), which imports the job modules undersrc/app/jobs(or the build output) and registers every concrete class.queue:workcalls it before working. Detection tests forhandleon the prototype rather thaninstanceof Job— a module loaded through jiti carries its own copy of the base class — and abstract bases are skipped.JobRegistryis backed by a global symbol, so a job module evaluated more than once registers into the one map the worker resolves against.Workergainson()handlers (onProcessing/onProcessed/onFailed, the last saying whether the job is being retried or has failed for good);queue:workprints each outcome.bootArkorm()now covers the--oncepath too.QueueContract.getDefaultQueue()exposes a connection's configured queue.Verification
Staged the built packages into the affected app and ran against an isolated Redis db:
loadJobs()against that app's build output discovers all 14 concrete job classes and skips its abstractAppJobbase; previously only the 2 that its bootstrap happened to import were resolvable.40 tests pass across
queue,jobsandscheduler, including new coverage forloadJobs(discovery, abstract/non-job/unloadable modules, missing directory) and for worker outcome reporting (retry → retry → permanent). Lint clean. The onetscerror inpackages/jobs/src/Job.ts:51is pre-existing, confirmed with these changes stashed.