Skip to content

feat(samples): community contribution flow + generated pipeline registry #556

Description

Summary

Epic: #555

First concrete deliverable of the community pipeline marketplace (epic: see linked issue). Stand up the contribution path + a generated, GitHub-hosted pipeline registry — without touching the server or UI yet. The output is a consumable, validated index.json (with a pipelines[] array) that a later phase wires into the server install flow.

Scope

  1. Contribution dir. Add a community/ location for community-submitted pipelines (decision in the epic: same-repo dir vs separate streamkit/pipelines repo). Each entry is a pipeline YAML plus the discovery metadata (group/variant/category/tags, landing via feat(samples): discovery UX with variant grouping and faceted search #544/Backfill explicit discovery metadata on sample YAMLs; thin the derivation heuristics #551) and a declared requires { plugins, models, features, assets }.
  2. CI validation (reuse, don't reinvent). Gate every submission with the existing checks: parse + compile + node-availability (the logic behind test(pipelines): add CI validation for official oneshot samples #542 / test(samples): guard dynamic samples with parse/compile + node-availability checks #543), so a contributed pipeline can't reference a nonexistent node kind or fail to compile. Validate that declared requires.plugins/features match the node kinds actually referenced in the YAML.
  3. Registry generator. Mirror scripts/marketplace/generate_official_plugins.py to emit per-version manifest.json for each pipeline and a registry index.json containing a pipelines[] array. Keep the catalog thin (metadata only); the YAML + requires{} live in the per-version manifest, fetched on demand.
  4. Publish. Build + publish the generated registry to GitHub Pages alongside the existing plugin registry (combined index.json vs dedicated URL — decision in the epic).

Schema (additive, backward-compatible)

RegistryIndex.plugins is already #[serde(default)] (apps/skit/src/marketplace.rs), so adding a sibling pipelines[] doesn't break existing registries or older servers. Pipeline manifest mirrors PluginManifest but carries inline yaml (or yaml_url), mode, discovery metadata, and requires{}. See the epic for the proposed shape.

Out of scope (phase 2, tracked off the epic)

  • Server-side list/fetch/install of pipelines and dependency resolution (marketplace.rs, marketplace_installer.rs, /api/v1/marketplace/pipelines*).
  • Any UI for browsing/installing pipelines.
  • Discovery/search service (Option C).

Acceptance

  • A PR adding a pipeline under community/ is validated by CI (parse/compile/node-availability + requires consistency).
  • The generator produces a signed-or-signable index.json with a populated pipelines[] and per-version manifests.
  • The registry is published and fetchable over HTTPS (GitHub Pages), gzip-compressed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions