Skip to content

test(e2e): automate Obsidian desktop sync flows with Playwright #115

Description

@ClaudiaFang

Context

#57 establishes real-provider/core E2E for SyncManager + production GitHub/GitLab/Gitea services. That layer should remain headless and deterministic.

This issue adds a second E2E layer for the actual Obsidian desktop experience: launch Obsidian, load the built plugin into an isolated vault, operate the plugin through its real GUI, and verify remote state against a real provider using an independent raw API verifier.

Goal

Verify that a user can complete the critical sync workflow through the real Obsidian GUI, not only through service classes.

The GUI suite must prove both sides of the boundary:

Playwright
  -> real Obsidian Desktop
  -> plugin UI / commands / settings
  -> SyncManager
  -> production provider service
  -> real Git provider
  -> independent raw API verification

A success toast alone is never sufficient evidence of a successful remote operation.

Dependency

Scope v1

Desktop harness

  • Launch an isolated Obsidian desktop instance from Playwright/Electron.
  • Create/use a disposable test vault per run.
  • Install the current plugin build (main.js, manifest.json, styles.css) into the test vault.
  • Enable the plugin automatically for the test vault.
  • Wait for Obsidian + plugin readiness using deterministic UI/state checks rather than arbitrary sleeps.
  • Capture screenshot + Playwright trace on failure.
  • Remove the disposable vault after a successful run; allow an opt-in keep-on-failure mode for debugging.

GUI smoke contract — all providers

Run the same minimal user workflow against GitHub, GitLab, and Gitea:

  • Open plugin settings.
  • Select/configure provider.
  • Configure repository/project, branch, root path, and token/credentials.
  • Execute Test Connection through the GUI and assert the visible result.
  • Create a note in Obsidian.
  • Trigger Push through the real plugin UI/command.
  • Verify the remote file through an independent raw provider API.
  • Mutate the remote file through the independent provider helper.
  • Trigger Pull through the real plugin UI/command.
  • Verify the local Vault/editor content changed to the remote content.

Canonical GUI provider — GitHub

Use GitHub as the exhaustive GUI provider after the cross-provider smoke contract is stable:

  • push new file;
  • push unchanged file and prove no extra remote commit;
  • pull remote update;
  • rename/move a note and verify old path removal + new path creation;
  • delete a note and verify remote deletion;
  • batch push path;
  • conflict protection flow;
  • plugin reload preserves settings and sync metadata;
  • relevant command palette / context menu entry remains functional;
  • error states surface actionable UI feedback.

Do not multiply every exhaustive GUI case across all three providers unless a case is genuinely provider-specific.

Provider matrix

Layer GitHub GitLab Gitea
Provider/core contract (#57) Full Full Full
SyncManager real-provider (#57) Full Full Full
GUI smoke Full Basic Basic
GUI exhaustive Canonical No No

Provider-specific API semantics continue to belong in #57. GUI tests are responsible for user-visible wiring and Obsidian integration.

High-value regressions this layer should catch

  • settings page fails to render on a supported Obsidian version;
  • provider selector/settings are not propagated into the production service;
  • command/ribbon/context-menu registration breaks;
  • modal or confirmation wiring blocks an operation;
  • Vault create/rename/delete event ordering causes an incorrect sync action;
  • plugin reload loses persisted configuration or sync metadata;
  • UI reports success although the remote repository was not changed;
  • remote update succeeds but the Obsidian editor/Vault is not refreshed;
  • Obsidian API changes break plugin initialization while unit/core E2E remain green.

Test architecture

Suggested structure:

e2e-gui/
├── playwright.config.ts
├── fixtures/
│   ├── obsidian-app.ts
│   ├── test-vault.ts
│   └── plugin-build.ts
├── pages/
│   ├── settings-page.ts
│   └── sync-ui.ts
├── suites/
│   ├── provider-smoke.spec.ts
│   └── github-full.spec.ts
└── helpers/
    └── diagnostics.ts

Prefer semantic selectors (getByRole, accessible name, stable data-* hooks where needed) over brittle CSS/layout selectors.

Credentials and sandboxing

CI strategy

Phase-in after local/manual stability:

  1. local GUI E2E against GitHub;
  2. manual workflow_dispatch for all providers;
  3. GitHub GUI smoke on internal PRs touching UI/sync/provider integration;
  4. GitLab/Gitea GUI smoke on main/manual/scheduled runs as stability permits.

Do not make the entire three-provider GUI matrix a required PR check initially. Keep #57's headless/core suite as the deterministic release gate; promote GUI checks only after flake rate is acceptable.

Acceptance criteria

  • A clean machine can launch an isolated Obsidian test vault and load the current plugin build.
  • GitHub/GitLab/Gitea GUI smoke can each complete settings -> connection -> push -> remote verify -> remote mutate -> pull -> local verify.
  • GitHub exhaustive GUI cases cover rename, delete, conflict, reload, and batch paths.
  • Remote assertions use the independent verifier from outside the plugin/service under test.
  • A visible success message cannot make a test pass if remote state is wrong.
  • Failures preserve useful screenshot/trace diagnostics without leaking credentials.
  • Existing headless/unit/provider E2E remain independent and runnable without Obsidian desktop.

Non-goals

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions