docs: bring the documentation in line with what the code does - #4
Open
timlandgraf wants to merge 5 commits into
Open
docs: bring the documentation in line with what the code does#4timlandgraf wants to merge 5 commits into
timlandgraf wants to merge 5 commits into
Conversation
The root README was a single title line, and viewer/README.md still described a ground-truth inspector — the prediction overlay, evaluation dashboard, re-clustering, Optuna search and GT editing were all undocumented. There was no reference for config.yaml at all, and nothing describing scripts/. - README.md: what the model is, the window -> average -> cluster pipeline, the window-level vs dance-level distinction, repo layout, setup, data layout and annotation schema, training, evaluation, tests. - viewer/README.md: rewritten for what the tool does now. Adds every CLI flag, the three prediction modes, the eval dashboard, the inspector and model view, and the GT override sidecar. Completes the keyboard table (P, Z, M, I, N, Delete, Escape were missing). - configs/README.md: new. Every post_process and eval key, why the eval config has two levels, what bee_size_multiplier does, and the two keys that are read by nothing (logging.checkpoint_dir, data.subset_size). - scripts/README.md: new. What each script is for, including the hardcoded paths in analyze_balance.py and the hardcoded video in propagate_gt.py. - combcell_annotator/README.md: the config section was wrong on nearly every value (data_path, n_videos, n_frames, a rotation option that no longer has any effect) and the output filename had a timestamp it never had. Adds projected_summary.csv, verify_projection.py, the S key, and where the measurement actually ends up (BEE_LENGTH_FRACTION). - combcell_annotator/docs.txt: dated note that /mnt/aglandgraf/wdd is no longer readable from either machine, so those checks cannot be redone as written. The conclusions still stand.
Hardcoded ckpt/noobj_rebalance_v2/best.pth; the current run is clean_split_v1. Made the checkpoint an argument with that default, and overridable via CHECKPOINT/PORT/DEVICE/EXTERNAL_VIDEOS. Also: fall back to ../.venv when the venv lives outside the checkout, warn instead of silently starting without predictions when the checkpoint is missing, and skip --external-videos when the directory is not mounted rather than passing a dead path.
"0:" is empty, from a mistyped redirect. "1" is 239 KB of captured server stdout from an April run — its contents are stale too (440 base recordings, 588/66 split; currently 160 and 582/72).
The test suite imports pytest but it was not declared, so `pytest tests/` failed on a fresh environment. Drops the install-it-first note from the README, which is no longer needed.
Line 12 imported VideoYoloDataset from src.data.dataset_tempaug, which exports VideoYoloDatasetTemporalJitter — the class was renamed and this test never followed. The correct import was sitting commented out directly above it. One collection error aborts the entire pytest run, so this single line was blocking all 54 tests. With it fixed the suite collects and runs: 53 passed, 1 failed. The remaining failure is test_augmentations.py::test_window_augmentations, which is unrelated: it reads a hardcoded /home/prajna/complete_data/annotations/... path and cannot pass anywhere except on that machine.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The repo grew a second half — the viewer, dance-level evaluation, post-processing, bee-size calibration — that was never written down, and a few documented facts have since become false. This is a documentation pass over all of it, plus two small fixes.
What was wrong
restart_server.shloadedckpt/noobj_rebalance_v2/best.pth; the current run isclean_split_v1.combcell_annotator/README.mdhad nearly every config value wrong:data_path: ../data(actually../../../data),n_videos: 2/n_frames: 3(actually 1 / 1), arotation:option that no longer has any effect since rotation is fixed at 0 in the code, no mention ofsave_csv, and an output filenameannotations_<shape>_<timestamp>.csvthat never had a timestamp.docs.txtsaid the raw footage on/mnt/aglandgraf/wdd/was readable except for the berlin files. The whole directory is nownobody:nogroupand unreadable from horus and goober — same NFS id-mapping fault asbeesbook_trajectory_data(INC-20055710).0:(empty, mistyped redirect) and1(239 KB of captured server stdout, itself stale — it reports 440 base recordings and a 588/66 split, currently 160 and 582/72).What was missing
README.mdwas one line. Now covers the model and the window → average → cluster pipeline, the window-level vs dance-level distinction, repo layout, setup (including that PyPI torch is CPU-only and fails silently), the data layout and annotation schema, training, evaluation and tests.viewer/README.mdstill described a ground-truth inspector. Nothing about--checkpoint/--device/--external-videos, the Raw/Averaged/Clustered prediction modes, the/evaldashboard, re-clustering, the Optuna search, the pipeline inspector, model view, or the non-destructive GT override sidecar. Its keyboard table was correct but missingP,Z,M,I,N,DeleteandEscape.configs/README.md(new) — there was no reference forconfig.yamlat all. Documents everypost_processkey, whyevalhas two levels with different threshold units, whatbee_size_multiplierdoes and why it is one global value, how to restorepost_process_baseline_tim.json, and the two keys read by nothing (logging.checkpoint_dir,data.subset_size).scripts/README.md(new) — five scripts, no documentation. Includes the caveats:analyze_balance.pyhas three absolute paths to/home/landgraf/..., andpropagate_gt.pyis hardcoded to one sharoni recording.Notes
Every claim was checked against the code at 587b07e rather than carried over — that is how the wrong combcell config values and the unused config keys turned up. Two things worth a second look:
restart_server.shbinds0.0.0.0, which I kept, but it does expose the annotation-editing endpoints to anything that can route to the machine. Flagged in the viewer README rather than changed.pytestwas missing fromrequirements.txteven though the suite imports it — now added (4th commit).src/tests/test_dataset.py:12importedVideoYoloDatasetfromsrc.data.dataset_tempaug, which exportsVideoYoloDatasetTemporalJitter— the class was renamed and the test never followed, with the correct import left commented out directly above. A single collection error aborts the whole pytest run, so this one line was blocking all 54 tests. Fixed in the 5th commit; the suite now runs: 53 passed, 1 failed.test_augmentations.py::test_window_augmentations, is left alone: it reads a hardcoded/home/prajna/complete_data/annotations/fps_multires_full_data.csvand cannot pass on any other machine. Same class of problem as the absolute paths inscripts/analyze_balance.py.@pytest.mark.slowis used intest_eval_utils_fast.pybut never registered, so every run emitsPytestUnknownMarkWarning. A shortpytest.iniwould silence it and make-m "not slow"actually work.Three commits, independently droppable: the documentation, the
restart_server.shfix, and the junk-file removal.🤖 Generated with Claude Code