Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/preview.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,17 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install dependencies
run: uv sync --all-groups
- name: Install locked dependencies
run: |
uv sync --locked --all-groups
npm ci --ignore-scripts
- name: Build generated assets
run: make build
- name: Verify Cloudflare auth
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
run: npx --yes wrangler@4.90.0 whoami
run: npm exec -- wrangler whoami
- name: Sync Python Workers vendor
run: uv run pywrangler sync
# Workflow inputs are passed through env vars rather than interpolated
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Install dependencies
run: uv sync --all-groups
- name: Pin Wrangler for local Worker runtime
run: npm install --global wrangler@4.90.0
- name: Install locked dependencies
run: |
uv sync --locked --all-groups
npm ci --ignore-scripts
- name: Start local Worker for browser checks
run: |
uv run --group workers pywrangler dev --port 9696 > /tmp/pythonbyexample-ci.log 2>&1 &
Expand All @@ -50,6 +50,7 @@ jobs:
env:
CHROME_PATH: /usr/bin/google-chrome
run: |
npm audit --audit-level=high
make verify
scripts/format_examples.py --check
make verify-python-version VERSION=3.13
Expand Down
6 changes: 4 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,12 @@ make build

## Before opening a pull request

Start the local Worker for browser-backed checks:
Install the committed dependency locks and start the local Worker with the repository-pinned Wrangler:

```bash
uv run --group workers pywrangler dev --port 9696
uv sync --locked --all-groups
npm ci --ignore-scripts
make dev
```

Then run:
Expand Down
12 changes: 9 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
# All Python tooling runs through uv pinned to 3.13 so the suite works on any
# machine regardless of the system python3 (examples use 3.12+ `type` syntax).
PY := uv run --python 3.13
NODE_DEPS_STAMP := node_modules/.package-lock.json

.PHONY: test embed-examples embed-editorial-registry build-search-index build check-generated fingerprint prototypes browser-layout-test search-ranking-test social-cards check-social-cards seo-cache-lint verify-examples check-registry-integrity check-confusable-pairs check-broad-surface-tours check-footgun-coverage check-notes-supported check-program-covers-cells check-prose-duplication check-inline-links score-example-criteria check-quality-scores check-no-figure-rationales check-journey-outcomes audit-example-graph quality-checks rubric-audit format-examples verify-python-version verify smoke-deployment dev deploy lint
.PHONY: node-deps test embed-examples embed-editorial-registry build-search-index build check-generated fingerprint prototypes browser-layout-test search-ranking-test social-cards check-social-cards seo-cache-lint verify-examples check-registry-integrity check-confusable-pairs check-broad-surface-tours check-footgun-coverage check-notes-supported check-program-covers-cells check-prose-duplication check-inline-links score-example-criteria check-quality-scores check-no-figure-rationales check-journey-outcomes audit-example-graph quality-checks rubric-audit format-examples verify-python-version verify smoke-deployment dev deploy lint

$(NODE_DEPS_STAMP): package.json package-lock.json
npm ci --ignore-scripts

node-deps: $(NODE_DEPS_STAMP)

test:
$(PY) python -m unittest discover -s tests -v
Expand Down Expand Up @@ -104,12 +110,12 @@ lint:

verify: build test seo-cache-lint verify-examples quality-checks browser-layout-test search-ranking-test lint check-generated

dev:
dev: node-deps
uv run --group workers pywrangler dev --port 9696

smoke-deployment:
$(PY) scripts/smoke_deployment.py $(URL)

deploy: check-generated
deploy: node-deps check-generated
uv run --group workers pywrangler sync
uv run --group workers pywrangler deploy
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,11 @@ This project is developed with red-green-refactor TDD:
2. Implement the smallest change that makes it pass.
3. Refactor while keeping tests green.

Install dependencies with `uv`, then run:
Install the exact committed Python and Node dependency sets, then run:

```bash
uv sync --locked --all-groups
npm ci --ignore-scripts
make test
```

Expand All @@ -81,10 +83,10 @@ After cloning, install the local git hooks once so merges and rebases regenerate
./scripts/install-git-hooks.sh
```

Run locally on Workers:
Run locally on Workers with the repository-pinned Wrangler:

```bash
uv run --group workers pywrangler dev --port 9696
make dev
```

Open:
Expand All @@ -98,7 +100,7 @@ http://localhost:9696
Run a local Worker before the full browser-backed verification:

```bash
uv run --group workers pywrangler dev --port 9696
make dev
```

Then run the main checks before deploying or pushing:
Expand Down Expand Up @@ -197,8 +199,8 @@ make check-generated

After adding an example or journey (or changing card title, summary, figure,
or card CSS), regenerate social cards. `make check-generated` verifies the
committed card-input provenance and exact JPEG set, without byte-comparing
Chrome's platform-variable raster output:
committed card-input provenance, exact JPEG set, JPEG decodability, and 1200×630
dimensions without byte-comparing Chrome's platform-variable raster output:

```bash
make social-cards
Expand Down Expand Up @@ -228,4 +230,6 @@ Use the active Cloudflare-supported Python version.
- The parent Worker uses a `LOADER` Worker Loader binding.
- Dynamic Worker IDs include Python version, example slug, and submitted code hash.
- Dynamic Workers run with `globalOutbound: null` and tight CPU/subrequest limits.
- POST example runs are never cached.
- POST example runs carry `Cache-Control: no-store` and are never cached.
- Wrangler is an exact dev dependency in `package-lock.json`; `make dev` and `make deploy` install/use that local version through Pywrangler.
- Production still requires the account-level WAF/rate-limit rule documented in `docs/turnstile-runner-protection-spec.md`; repository settings alone do not prove that external control is active.
8 changes: 3 additions & 5 deletions docs/example-graph-see-also.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,12 @@ see_also = [
]
```

The loader exposes `see_also`, the example renderer displays a compact `See also` section, and tests verify that every linked slug exists and does not point to itself.
The loader exposes `see_also`, the example renderer displays a compact `See also` section with registry-backed edge labels such as `contrast` and `next depth`, and tests verify that every linked slug exists and does not point to itself. The same graph supplies bounded recommendations on missing-example pages.

## Graph audit

`scripts/audit_example_graph.py` reports orphan pages, high in-degree pages, and reciprocal links. Its `--check` mode runs in `make quality-checks` and fails the build on broken targets, self-links, out-degree above 4, or orphaned examples.

## Future improvements
## Review rule

- Show short edge labels later, e.g. `contrast`, `prerequisite`, `next depth`.
- Use graph data to recommend examples on 404 pages or search results.
- Keep the first version minimal until we know the links improve reading rather than distracting from the code.
Keep labels and recommendations minimal. Each edge still needs editorial review: passing graph checks proves structural integrity, not that a link improves the lesson.
62 changes: 36 additions & 26 deletions docs/learner-analytics.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,27 @@
# Learner analytics

The Worker emits one structured wide event per request (see
`docs/observability-spec.md`). Those events already carry the fields
that matter for content decisions: the page path, the example slug on
POST runs, whether the submitted code was edited, the Dynamic Worker
outcome, and the execution time.

`scripts/learner_report.py` turns an export of those events into a
learner-behavior report so content work is steered by external signal
rather than internal quality scores:

- **Most-read pages** — GET traffic per path.
- **Most-run examples** — POST runs per slug, with the edited share
(are people experimenting or just pressing Run?), the error share
(where edited code fails), and p50/p95 execution times.
- **Journey traffic** — which curated arcs get used.
- **Missing-example requests** — 404s under `/examples/`, i.e. demand
for pages that do not exist yet. These are content candidates.
- **Turnstile outcomes** — how often runs are challenged or blocked.
`docs/observability-spec.md`). Those events carry the page path, the
example slug on POST runs, whether submitted code differed from the
example, the Dynamic Worker outcome, and execution time.

`scripts/learner_report.py` aggregates those events into an
**operational interaction scorecard**. It does not identify people,
deduplicate visits, measure comprehension, or prove that a content
change caused learning. Every row is a request count within the export's
time window:

- **Most-requested pages** — successful GET requests per path, including
repeat requests and possible automated traffic.
- **Most-run examples** — accepted POST attempts per slug, with edited,
error, and p50/p95 execution-time distributions. "Edited" means only
that submitted code differed; it is not evidence of successful learning.
- **Journey requests** — GET request volume for curated-arc routes.
- **Missing-example requests** — 404s under `/examples/`. Recurring,
human-validated paths may suggest content candidates, but typos, scanners,
and bots are confounders.
- **Turnstile outcomes** — challenge and rejection counts used to monitor
runner availability and abuse controls, not learner quality.

## Getting events

Expand All @@ -37,12 +41,18 @@ envelope, so exports from any of the three sources work unmodified.
`--json` emits the aggregated report as JSON for further processing;
`--limit N` controls rows per section.

## Reading the report

- A high **edited share** with a low error share means the example
invites successful experimentation — the ideal.
- A high **error share** on edited runs marks pages where learners try
something the example did not prepare them for; consider extending
the walkthrough or notes there.
- **Missing-example requests** that recur are the strongest possible
signal for what to write next.
## Reading the scorecard

Use the report as a queue for investigation, not as a composite score or
a learning-outcome dashboard. Record the export interval and absolute
sample size, compare equivalent time windows, and inspect paths or code
patterns before changing content.

- A high **edited share** can justify reviewing an example for useful
extension points. Pair it with error samples and direct learner feedback.
- A high **error share** can indicate unclear teaching, intentionally invalid
experiments, abuse, or runtime trouble. Diagnose those alternatives first.
- Recurring **missing-example requests** are candidates only after filtering
obvious automation and validating the intended topic.
- Do not rank authors or examples by one ratio, and do not claim learning
improvement without a separate outcome measure and an appropriate study.
Loading
Loading