Skip to content

fix(cypress): Chainlit shutdown and entrypoint checks - #2728

Open
coterp wants to merge 4 commits into
Chainlit:mainfrom
coterp:fix-cypress-chainlit-shutdown
Open

coterp wants to merge 4 commits into
Chainlit:mainfrom
coterp:fix-cypress-chainlit-shutdown

Conversation

@coterp

@coterp coterp commented Dec 17, 2025

Copy link
Copy Markdown

Summary

This PR improves the reliability of Cypress E2E runs when starting and stopping Chainlit.

Changes

  • Track the spawned Chainlit process and shut it down reliably via process group
  • Use port-based termination (fkill) only as a fallback when no tracked process exists
  • Fix broken entrypoint file existence check (async fs/promises.access was not awaited)
  • Improve signal handling and typing in cypress.config.ts
  • Modernize Node core imports (node: + type imports)

Motivation

Cypress E2E runs were failing consistently due to unreliable Chainlit shutdown and restart behavior in containerized dev environments.

Tests frequently failed with port-in-use errors because previous Chainlit instances were not reliably terminated between runs or between specs. The existing port-based termination (fkill(:port)) proved unreliable in containers and was insufficient as a primary means of terminating Chainlit.

This PR introduces explicit process tracking and shutdown of the spawned Chainlit process, using port-based termination only as a fallback to clean up stray instances from previous runs.

Testing

  • uv run pytest --cov=chainlit
  • pnpm test (Cypress, headless via Xvfb)
  • pnpm test:ui
  • pnpm lint

Summary by cubic

Cleanly starts and stops Chainlit during Cypress E2E runs to prevent port-in-use errors, especially in containers and on Windows. Previously we relied on port-based fkill and a non-awaited entrypoint check; now we track the spawned process and kill the process group (or entire tree on Windows) with verified startup.

  • Add stopChainlit() to terminate the tracked process; use fkill :8000 only as a fallback.
  • Spawn Chainlit as a detached process and retain currentChainlit; handle SIGTERM, SIGINT, SIGHUP, and SIGBREAK, then exit with the correct code after cleanup.
  • Windows: kill the entire process tree via taskkill /T /F and wait briefly to ensure port release; POSIX: send SIGTERM then SIGKILL to the process group.
  • Fix entrypoint check by awaiting fs/promises.access; throw a clear error if the file is missing.
  • Simplify restartChainlit with a flat promise chain and modernize node: imports and typed error handling.

Written for commit 42422c0. Summary will update on new commits.

Review in cubic

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. dev-tooling Relating to developer/contributor toolings. labels Dec 17, 2025

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files


Since this is your first cubic review, here's how it works:

  • cubic automatically reviews your code and comments on bugs and improvements
  • Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
  • Ask questions if you need clarification on any suggestion

@asvishnyakov

Copy link
Copy Markdown
Member

@coterp Thank you for contribution!

@hayescode @sandangel I would like to review this PR myself, as I tried to implement such functionality earlier and had an issues with that

@coterp

coterp commented Dec 17, 2025

Copy link
Copy Markdown
Author

@hayescode Sounds good, thanks for your review.
Let me know if you have any questions or want me to adjust anything.

@asvishnyakov asvishnyakov changed the title Fix Cypress Chainlit shutdown and entrypoint checks fix(cypress): Chainlit shutdown and entrypoint checks Dec 23, 2025

@asvishnyakov asvishnyakov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coterp Unfortunately I have same issue with it as when I tried implement such feature myself: it doesn't work in Cypress interactive mode, at least on Linux
Try to:

  • Open Cypress in interactive mode (for example, via pnpm test:interactive)
  • Run custom_theme tests (you'll see green screen, then red)
  • Run data_layer tests (you'll see green screen again and tests will fail because login isn't enabled in previous test server, while it is in data_layer test server)

@asvishnyakov

Copy link
Copy Markdown
Member

But I think we can merge some implements from here, like broken entrypoint file existence check, if you'll remove or fix new server restart code

@coterp

coterp commented Dec 30, 2025

Copy link
Copy Markdown
Author

Thanks for the reproduction steps and feedback. That helps clarify the scope.

Given that the restart behavior only affects Cypress interactive mode on Linux and isn’t something I can easily validate in my current dev container setup, I think the safest approach is to split this PR and drop the restart changes for now, keeping only the clearly correct fixes.

I should be able to push that split as early as tomorrow, though it may slip into early next week depending on availability.

@coterp

coterp commented Dec 30, 2025

Copy link
Copy Markdown
Author

I went ahead and tried to reproduce the issue in Cypress interactive mode on Linux at commit d8e9a68d. I ran custom_theme and then data_layer in the same Cypress session, and both passed for me. I tried this with both Electron and Chrome.

I’m running on RHEL 9.7. Could you share which browser and distro/desktop you’re using? I’ll try to match it.

@asvishnyakov

Copy link
Copy Markdown
Member

@coterp Happy holidays! I'm using Ubuntu 24.04 and running Cypress via pnpm test:interactive with Electron. May be it was something local, let me try again within few days and I'll return with additional feedback

@coterp

coterp commented Jan 6, 2026

Copy link
Copy Markdown
Author

Thanks! I hope you had happy holidays as well!

Quick update: I set up a test environment on Ubuntu 24.04 and still wasn’t able to reproduce the behavior you mentioned. I tried running the specs in rapid succession to rule out timing issues, tested with both Electron and Chrome, and ran several other specs, but couldn’t get a failure.

Let me know if there’s any additional detail I can try to match.

@coterp

coterp commented Jan 19, 2026

Copy link
Copy Markdown
Author

@asvishnyakov
Just wanted to check whether you had a chance to retry this on your side.

No rush, happy to wait or adjust if you still see issues.

@coterp
coterp requested a review from asvishnyakov February 16, 2026 23:13
@dokterbob dokterbob added the review-me Ready for review! label Feb 23, 2026
@pidefrem

pidefrem commented Jun 8, 2026

Copy link
Copy Markdown

Hi @coterp @asvishnyakov,

I landed here while investigating the same [Errno 48] address already in use failures on macOS (Apple Silicon, Node 26, Cypress 14.5.3). I can independently confirm the bug and that the process-group approach fixes it for cypress run (headless CI mode).

What I observed:

Running pnpm test:e2e on macOS, every spec after the first fails with:

[Chainlit stderr] ERROR: [Errno 48] error while attempting to bind on address ('127.0.0.1', 8000): [errno 48] address already in use

The root cause is the same as this PR identifies: fkill :8000 kills uvicorn (the socket owner) but leaves the parent uv run process alive. Spawning with detached: true and killing by process group (process.kill(-pid, 'SIGKILL')) resolves it completely — specs that previously failed after 4 retries (2 min each) now pass on the first attempt in under 2 seconds.

I also added a waitForPortFree() helper that polls net.createServer() on 127.0.0.1:8000 to confirm the port is actually released before starting the next server. This prevents any remaining OS-level socket teardown delay from causing a race.

access() dead-code bug also confirmed:

I also independently found the !access(entryPointPath) dead-code bug in cypress/support/run.ts — access() from fs/promises returns a Promise, so !Promise is always false and the entry-point check never fires. This PR fixes that too by properly awaiting it outside the new Promise() constructor.
So this PR addresses two independently verified bugs.

On the interactive mode concern:

@asvishnyakov — the issue you described (switching specs in interactive mode picks up the wrong server config) sounds like it might be a separate Cypress lifecycle concern rather than a problem with the kill/restart approach itself. coterp tried to reproduce it on both RHEL 9.7 and Ubuntu 24.04 without success. Would you be able to retry on a current main checkout? If the interactive mode issue persists, it could be addressed in a follow-up PR rather than blocking this fix for the cypress run path that CI relies on.

Suggestion:

This PR has been open since December and now has merge conflicts. Would it make sense to:

  1. Rebase on main to resolve conflicts
  2. Merge as-is for cypress run mode (all CI checks were green)
  3. Open a follow-up issue for any interactive mode regressions if they're still reproducible

Happy to help test or contribute if useful.

@coterp

coterp commented Jun 9, 2026

Copy link
Copy Markdown
Author

Thanks @pidefrem, I appreciate the independent confirmation and the extra testing details.

@asvishnyakov, I’m happy to rebase this on current main and resolve the merge conflicts. Since the cypress run failure path and the access() bug have now both been independently confirmed, would you be comfortable moving this PR forward and handling any remaining interactive-mode issue separately if it is still reproducible?

@dokterbob

Copy link
Copy Markdown
Collaborator

Thanks @pidefrem, I appreciate the independent confirmation and the extra testing details.

@asvishnyakov, I’m happy to rebase this on current main and resolve the merge conflicts. Since the cypress run failure path and the access() bug have now both been independently confirmed, would you be comfortable moving this PR forward and handling any remaining interactive-mode issue separately if it is still reproducible?

Is this still relevant? If so, I'm happy to approve and merge once you rebase. (Very little bandwidth to get into the nitty gritty... unfortunately!)

@coterp
coterp force-pushed the fix-cypress-chainlit-shutdown branch from d8e9a68 to 5235b41 Compare August 18, 2026 02:26
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Aug 18, 2026
@coterp

coterp commented Aug 18, 2026

Copy link
Copy Markdown
Author

@dokterbob Thanks for taking a look.

Yes, I believe this is still relevant. I’ve rebased the PR onto current main, resolved the conflict in cypress/support/run.ts, and re-ran the relevant tests. Headless E2E passes, and I also re-tested the original interactive-mode scenario (custom_theme followed by data_layer in the same Cypress session) successfully.

Thanks again for offering to review and merge it.

@coterp

coterp commented Aug 26, 2026

Copy link
Copy Markdown
Author

@dokterbob The Windows E2E failure appears to be unrelated to this PR. I checked adjacent CI runs and found the same oauth_auth/spec.cy.ts failure in CI #6339 on an unrelated PR and CI #6340 on main. Later runs #6342 and #6343 passed.

Could we rerun the failed CI jobs for this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dev-tooling Relating to developer/contributor toolings. review-me Ready for review! size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants