Skip to content

Repository files navigation

Grounded

Highly skeptical review skills for machine learning and data engineering.

Skill Asks
is-this-real What does this actually do, and is its name real outside this repo?

In testing: is-this-needed, what-goes-where, make-it-make-sense, review.


Why this exists

You take a look at your agent's output after its autonomous session. Or worse, you take a look at a PR you're supposed to review. Somewhere inside some enormous diff, you pick out a few names: a feature-lineage-conformance-contract-v4, a MATERIALIZE_OR_HYDRATE_POLICY enum, and a three-tier degraded-build fallback ladder.

None of this feels entirely real, even though everything looks plausible. There are no syntax errors, there are type hints everywhere, test coverage is confidently high, docstrings look beautiful, and there are no linters nagging.

From the first look you can't immediately tell that all of these changes sit on top of an algorithm that is not actually the algorithm its name claims, a subsystem nobody needs, a data flow nobody can trace, and vocabulary and concepts that exist nowhere outside your own repository.

You launch some default code review workflow of an AI coding agent of your choice, and, in the meantime, to get a better understanding of what these code changes could be referring to, you take a look at an accompanying markdown file, thirteenth of its kind slapped into docs/ by now.

The document, describing the design behind the changes, talks of delta-append build resolution for historical catch-up replay artifacts, generated on a cadence that is itself resolved by some resolve_train_cadence_decision method of a TrainCadenceDecision class, whose digest has to match the provenance_token sitting in a locally-saved json file emitted alongside snapshot_fidelity_contract generated by a build_snapshot_fidelity_fingerprint_factory, whose hashed payload includes, among eleven other fields, the aforementioned feature-lineage-conformance-contract-v4.

The document makes sure to warn you:

This feature-lineage-conformance-contract-v4 is load-bearing, with two honest caveats.

If you're lucky, there's a mermaid diagram somewhere after the 300 line mark of the doc with 16 nodes that you can't quite zoom into.

Meanwhile, your review agent returns with findings of impossible edge cases to snapshot_fidelity_contract validation at read-time and possible failure modes of the three-tier degraded-build fallback ladder, proposing an extra, fourth tier.

At some point you begin to wonder if we've lost the plot.

If any of that sounded vaguely reminiscent of what you've dealt with, you are not alone. The skills in this repo are what I use in my day-to-day to keep my work grounded in reality, one adversarial question at a time, reading what the code does before accepting what it is called, and treating its accompanying documentation as a claim rather than a source.

Install

Claude Code

/plugin marketplace add anasashb/grounded
/plugin install grounded@grounded

Codex

codex plugin marketplace add anasashb/grounded
codex plugin add grounded@grounded

Kiro

Install as skills

Install Grounded directly with Vercel Labs' skills CLI:

npx skills@latest add anasashb/grounded --agent kiro-cli --global

Kiro makes installed skills available automatically and as slash commands, such as /is-this-real.

Install as a Kiro Power

In the Kiro IDE:

  1. Open Powers and select Add Custom Power.
  2. Select Import power from GitHub.
  3. Enter https://github.com/anasashb/grounded and select Install.

Kiro CLI v3 automatically detects Powers installed through the IDE.

is-this-real

You are an ML Engineer, Data Scientist, or Data Engineer.

You look at a diff showing an OutputUncertaintyCalibrator, with three private methods and ten try-excepts wrapping a np.clip. Somewhere else in a diff there's an EnsembleStrategy class holding one model. A +427 diff on a markdown file talks of a "load-bearing" DailyFrequencyModelTrainingOrchestrator class, which, upon inspection, seems to be a for loop wrapped with a try-except.

Or, you're reviewing a branch called feature/outcome-maturity-reconciliation and its new code and documentation to find out that what they mean by this is just plain model evaluation with delayed labels. Maybe, somewhere in there there's a LossAdapterResolutionPolicy doing god-knows-what. An accompanying YAML file defines config-driven steps, fifth of which is called entity_resolution_stage, in reality doing a simple join between two dataframes.

Are any of these concepts in code, config or docs real?

Probability calibration is a real thing that exists in ML literature, but if the agent implements a np.clip below 1.0 and calls it OutputUncertaintyCalibrator, then it's wrong. It goes the other way too. You look at an ArtifactRetentionGovernancePolicy class, convinced that the AI made it up, and it turns out to wrap a plain TTL method under a confusing name. And when your agent implements a custom, in-house business rule for filtering data or outputs, it will "package it" as something so formal that one would think this is a widely-adopted method straight from some data engineering textbook.

The is-this-real skill tries to address these issues by making the AI agent review the code by reading the implementation first, and challenging existing naming, methods and documentation without trusting what the repository claims it does blindly.

It looks into three issues (and conducts comprehensive web searches):

  • Where did this concept come from? Is it established in the field, adapted from something established, or local to this repository?
  • Does the implementation fit the concept? Does the code have the concept's defining properties, only some of them, or does the name promise behavior that the code does not provide?
  • What about the claims written around the concept - e.g. that a threshold is tuned, that an approach is standard practice, or that some benchmark was met. Does the evidence support the claims made by the code and/or its documentation?

A simplified example of what is-this-real judges:

k-fold validation as a simplified example of the axes is-this-real judges on.

Local names, or in-house logic and concepts are not a problem. The skill will not call a choice wrong only for being local, but will push back on it if it uses a name that is established for something else (even if similar).

This skill relies heavily on external sources and web search and does not answer from memory. If it cannot find authoritative evidence, it will report this or that claim as unverified.

About

Highly skeptical review skills for machine learning and data engineering.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Contributors

Languages