[RFC]: Static type checking — explore Pyrefly integration for Skillware

[rosspeili]

Summary

Open discussion on whether Skillware should adopt static type checking in CI and contributor workflow, with Pyrefly as the primary candidate to explore (not a commitment to ship).

Skillware today enforces black, flake8, and pytest only — there is no mypy, pyright, or Pyrefly in pyproject.toml, CI, or contributor docs. Core modules (skillware/core/, cli.py) already use partial typing annotations; registry skills vary in annotation depth and optional-dependency patterns.

This RFC does not implement anything. It aligns on whether type checking is worth the contributor cost, what scope to check if yes, Pyrefly vs alternatives, and how to roll out without blocking skill-only PRs.

Motivation

Problem

• Framework code (loader, discovery, adapters, CLI) is growing in complexity ([Feat]: CLI commands to set and repair skill directory paths (sub-issue of #16) #81 discovery module, future [Feat]: CLI config and three-tier skill path resolution (global + project) #246 config paths, [Feat]: Skill load trust tiers — origin metadata and warnings for non-bundled skills (sub-issue of #17) #110 trust warnings). Type errors in these paths are caught late — at runtime or via tests — not at edit/CI time.
• Contributors and agents follow black + flake8 + pytest; there is no static contract for public APIs (SkillLoader, BaseSkill, bundle shape).
• The Python tooling landscape shifted in 2026: Pyrefly 1.0 (Meta) offers fast checking, gradual presets (basic, legacy), and migration helpers — but Skillware has no existing mypy investment to migrate from; this is a greenfield decision.

What we want from this RFC

1. Go / no-go / phased — Is static typing worth adding to Skillware's quality bar?
2. Tool choice — If yes, is Pyrefly the right first explore (vs mypy, pyright, ty, or defer)?
3. Scope — Framework only (skillware/, tests/) vs entire repo including skills/**?
4. Rollout — Optional dev extra vs enforced CI; warn-only vs blocking; baseline/suppress strategy.
5. Ripple effects — CONTRIBUTING, TESTING.md, PR template, CI time, contributor friction for skill authors.

Non-goals

• Mandating full strict typing on every registry skill in one release.
• Replacing flake8 or black.
• Blocking [RFC]: Skill identity and registry architecture #234 / [RFC]: Install model — one wheel, category and skill dependency extras #236 / skill shipping on this RFC.

Detailed Design

Current state

Area	Today
CI	black --check, flake8, pytest skills/, pytest tests/
Type checker	None configured
Annotations	Partial in skillware/; inconsistent across skills/
Contributor docs	TESTING.md, PR template — no type-check step

Questions to resolve (open)

1. Is it worth it?

Pros of adding static type checking

• Catches API/shape bugs in loader, discovery, CLI before merge.
• Documents public contracts for contributors and agent workflows.
• Better IDE/LSP experience (Pyrefly ships a language server).
• Low incremental CI time on a repo this size if scoped to framework code.

Cons / reasons to defer

• Extra contributor step; skill-only PRs may not benefit.
• skills/ use lazy imports, optional deps (web3, rembg, SDK extras) — whole-tree strict checking is noisy without investment.
• Another tool to document, baseline, and maintain.
• Pyrefly is production-proven but younger than mypy; plugin/stub ecosystem still maturing.

2. Why explore Pyrefly first (if we explore at all)?

Skillware has no mypy config to migrate — tool choice is greenfield.

Consideration	Pyrefly	mypy	pyright
Speed (large codebases)	Strong	Weaker; 2.0 parallel helps	Moderate
Greenfield adoption	basic preset, pyrefly init, suppress	Mature, widest ecosystem	Strong conformance
Aggressiveness	Can flag more on unannotated code	Gradual by default	Strict options
Skillware fit	Good for phased skillware/ scope	Safe default if stub/plugins needed	Alternative spike

Recommendation for spike (not decision): evaluate Pyrefly on skillware/ + tests/ with basic or legacy preset; compare error signal vs noise against a short mypy baseline on the same paths.

3. Scope options (proposed tiers)

Tier	Paths	CI	Notes
A — Defer	—	—	Status quo
B — Dev optional	skillware/, tests/	No	Document in TESTING.md; pip install pyrefly; contributor opt-in
C — CI advisory	skillware/, tests/	Non-blocking	Report in CI; fix baseline over time
D — CI required	skillware/, tests/	Blocking	Full gate after baseline clean
E — Skills included	+ skills/**	TBD	High friction; likely optional/warn-only for skill authors

Default RFC bias: land between B and C first; E only if explicitly valued and phased like #200 (warn-only identity checks).

4. What to consider when adding it

Technical

• Preset choice: basic (low noise) vs default vs strict.
• Baseline: pyrefly suppress or fix-forward for existing annotations.
• Optional deps / dynamic imports in loader and skills — may need type: ignore or stub packages.
• Windows + Linux CI (Pyrefly supports both).
• Interaction with [dev] extra vs standalone install.

Process

• Update: pyproject.toml, .github/workflows/ci.yml, docs/TESTING.md, CONTRIBUTING.md, PR template checklist.
• Do not require skill contributors to pass Pyrefly until tier E is explicitly agreed.
• Changelog entry when CI behavior changes (user-visible for contributors).

Example spike commands (illustrative)

pip install pyrefly
pyrefly check skillware/
pyrefly check tests/

5. Suggested spike deliverables (follow-up issue, not this RFC)

• Run Pyrefly + mypy on skillware/ and tests/; compare error counts and false positives.
• Draft pyproject.toml / config fragment (if adopt).
• Decision record appended to this RFC: adopt / defer / partial.
• If adopt: implementation issue for chosen tier (B/C/D).

6. Child / related issues (to create after RFC closes)

Follow-up	Title (draft)
Implementation	[Feat]: Add Pyrefly type checking for skillware/ (phased)
Docs	Update TESTING.md + CONTRIBUTING contributor checklist
Optional	Type annotations hardening in loader/discovery

Suggested RFC outcomes (when ready to close)

• Agreed go / no-go / phased for static type checking
• If go: agreed scope (framework-only vs skills) and CI tier (optional / advisory / required)
• Tool recommendation: Pyrefly vs mypy vs defer — with short rationale
• Contributor-impact statement (what skill authors must vs may do)
• Spike issue created or explicitly declined

Drawbacks

• Process overhead — RFC + spike + possible baseline PR delays other framework work.
• False sense of security — Types do not sandbox skills; orthogonal to trust model ([RFC]: Skill identity and registry architecture #234, skill-trust-model.md).
• Contributor friction — Even framework-only CI gates add a step; may deter drive-by skill PRs if scope creeps to skills/.
• Tool churn — Python type-checker landscape is moving fast (Pyrefly, mypy 2.0, ty); committing too early may require rework.
• Maintenance — Baseline suppressions and stub gaps become ongoing debt if not kept tight.
• Opportunity cost — [Feat]: Loader dependency diagnostics and load modes #235, [RFC]: Install model — one wheel, category and skill dependency extras #236, [Feat]: Skill load trust tiers — origin metadata and warnings for non-bundled skills (sub-issue of #17) #110 may deliver more user-visible value than type CI in the next release.

Mitigation: Keep RFC non-binding; spike is time-boxed; default scope is skillware/ + tests/ only; defer skills/ enforcement unless explicitly approved.