Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ uv run ruff check .
uv run ruff format --check src tests examples tools
uv run qf bench
uv run qf audit
uv build
```

## Adding an adapter
Expand All @@ -40,3 +41,14 @@ Implement `catalog`, `read_series`, `tables`, `read_table`, `invariants` and
against it — see `tests/test_duckdb_adapter.py` for the pattern. An adapter
that cannot serve `pub_date` per observation cannot support point-in-time and
should say so in its docstring rather than pretending.

Run `qf adapter-check --json adapter-conformance.json` before writing bespoke
integration tests. Passing the suite is necessary, not proof that a data source
is accurate or licensed.

## Claims and operating evidence

Do not describe repository functionality as expert accuracy, production safety
or user value. Changes to `benchmarks/quality-evidence.json` require an immutable
raw artifact, measurement window, denominator, sample size and named approver.
See `docs/acceptance.md` and `ROADMAP.md` before proposing a maturity claim.
2 changes: 2 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
- [ ] `uv run ruff format --check src tests examples tools` passes
- [ ] `uv run qf audit` still reports evidence and gaps accurately
- [ ] no credentials, licensed data or workspace directories committed
- [ ] maturity and security language does not overstate what the evidence proves
- [ ] generated site and evidence/package examples still reproduce when relevant

## If this touches contracts or the harness
- [ ] the property it guarantees is stated in the test name
Expand Down
14 changes: 14 additions & 0 deletions .github/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,17 @@ treat the workspace directory as untrusted output.
The core ships no credentials and no data. Adapters read theirs from the
environment. Never commit an API key, a database path containing licensed data,
or a workspace directory; `.gitignore` covers the defaults and CI scans commits.

## Evidence-package integrity

`qf verify` checks internal hashes and manifests. It is not publisher
authentication: anyone able to replace an entire package can recompute its
self-hash. A production deployment should sign package identities or publish
them through a trusted transparency registry.

## Supported versions

Quantifact is pre-1.0 alpha software. Security fixes are applied to the latest
commit on `main`; older source snapshots are not maintained. There is currently
no production-safe release because process/container isolation is not yet part
of the core runtime.
14 changes: 14 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
version: 2
updates:
- package-ecosystem: uv
directory: /
schedule:
interval: monthly
open-pull-requests-limit: 5
labels: [dependencies]
- package-ecosystem: github-actions
directory: /
schedule:
interval: monthly
open-pull-requests-limit: 5
labels: [dependencies]
41 changes: 41 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ jobs:
git diff --exit-code -- site/index.html
- name: Point-in-time example
run: echo n | uv run python examples/04_point_in_time/run.py
- name: Adapter conformance
run: uv run qf adapter-check --json /tmp/adapter-conformance.json
- name: Sliced research evaluations
run: uv run qf evals --dir benchmarks --json /tmp/research-evals.json

build:
runs-on: ubuntu-latest
Expand All @@ -50,8 +54,45 @@ jobs:
uv venv --python 3.12 /tmp/qf-wheel-test
uv pip install --python /tmp/qf-wheel-test/bin/python dist/quantifact-*.whl
cd /tmp
/tmp/qf-wheel-test/bin/qf --workspace /tmp/qf-wheel-run ask \
--out /tmp/qf-wheel-report.html \
--evidence /tmp/qf-wheel-evidence.json
/tmp/qf-wheel-test/bin/qf verify /tmp/qf-wheel-evidence.json
/tmp/qf-wheel-test/bin/qf adapter-check --json /tmp/qf-wheel-adapter.json
/tmp/qf-wheel-test/bin/qf audit --json /tmp/qf-wheel-audit.json
set +e
refusal=$(/tmp/qf-wheel-test/bin/qf plan "Build a DCF valuation" 2>&1)
status=$?
set -e
test "$status" -eq 2
test "$(printf '%s\n' "$refusal" | wc -l)" -eq 1
printf '%s\n' "$refusal" | grep '^refused: '
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/

docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Check local Markdown links
run: |
python - <<'PY'
from pathlib import Path
import re

roots = [Path("."), Path("docs"), Path(".github"), Path("examples"), Path("benchmarks")]
files = sorted({p for root in roots for p in root.rglob("*.md") if ".git" not in p.parts})
missing = []
for path in files:
for target in re.findall(r"\[[^]]*\]\(([^)]+)\)", path.read_text()):
target = target.split("#", 1)[0]
if not target or "://" in target or target.startswith("mailto:"):
continue
if not (path.parent / target).resolve().exists():
missing.append(f"{path}: {target}")
if missing:
raise SystemExit("broken local links:\n" + "\n".join(missing))
print(f"checked {len(files)} Markdown files")
PY
92 changes: 80 additions & 12 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,93 @@ on:
push:
tags: ["v*"]

permissions:
contents: read
id-token: write # PyPI trusted publishing

jobs:
publish:
validate-build:
runs-on: ubuntu-latest
environment: pypi
permissions:
contents: read
id-token: write
attestations: write
outputs:
version: ${{ steps.version.outputs.version }}
prerelease: ${{ steps.version.outputs.prerelease }}
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
- run: uv sync --extra duckdb
- name: Validate release identity
id: version
run: |
uv run python tools/check_release.py --tag "$GITHUB_REF_NAME"
version=$(uv run python tools/check_release.py --print-version)
echo "version=$version" >> "$GITHUB_OUTPUT"
if [[ "$version" =~ (a|b|rc|dev)[0-9]+ ]]; then
echo "prerelease=true" >> "$GITHUB_OUTPUT"
else
echo "prerelease=false" >> "$GITHUB_OUTPUT"
fi
- run: uv run ruff check .
- run: uv run ruff format --check .
- run: uv run pytest -q
- name: Require audited operating evidence
run: test -f benchmarks/quality-evidence.json
- name: PAT-level release gate
run: uv run qf audit --evidence benchmarks/quality-evidence.json --strict
- run: uv run qf evals --dir benchmarks --json /tmp/research-evals.json
- run: uv run qf adapter-check --json /tmp/adapter-conformance.json
- name: Stable/PAT-level operating-evidence gate
if: steps.version.outputs.prerelease == 'false'
run: |
test -f benchmarks/quality-evidence.json
uv run qf audit --evidence benchmarks/quality-evidence.json --strict
- name: Build clean distributions
run: uv build
- name: Smoke-test installed wheel
run: |
uv venv --python 3.12 /tmp/qf-release-test
uv pip install --python /tmp/qf-release-test/bin/python dist/quantifact-*.whl
cd /tmp
/tmp/qf-release-test/bin/qf --workspace /tmp/qf-release-run ask \
--out /tmp/qf-release-report.html \
--evidence /tmp/qf-release-evidence.json
/tmp/qf-release-test/bin/qf verify /tmp/qf-release-evidence.json
- uses: actions/attest-build-provenance@v2
with:
subject-path: "dist/*"
- uses: actions/upload-artifact@v4
with:
name: release-dist
path: dist/

github-release:
needs: validate-build
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/download-artifact@v4
with:
name: release-dist
path: dist
- name: Publish GitHub release
env:
GH_TOKEN: ${{ github.token }}
PRERELEASE: ${{ needs.validate-build.outputs.prerelease }}
VERSION: ${{ needs.validate-build.outputs.version }}
run: |
rm -rf dist
uv build
args=(--repo "$GITHUB_REPOSITORY" --verify-tag --generate-notes \
--title "Quantifact $VERSION")
if [[ "$PRERELEASE" == "true" ]]; then
args+=(--prerelease)
fi
gh release create "$GITHUB_REF_NAME" dist/* "${args[@]}"

publish-pypi:
if: needs.validate-build.outputs.prerelease == 'false'
needs: validate-build
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v4
with:
name: release-dist
path: dist
- uses: pypa/gh-action-pypi-publish@release/v1
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,33 @@ All notable changes to this project are documented here, following

## [Unreleased]

## [0.3.0a1] — 2026-08-13

This alpha release strengthens the repository's fail-closed research and
distribution boundaries. It does not claim expert validation, production
isolation, or investment fitness.

### Added
- Versioned `ResearchEvidencePackage` with plan, code, source-vintage manifest,
claim lineage, value fingerprints, admission semantics and integrity checks.
- Offline `qf verify` command with explicit integrity/authenticity boundary.
- Registered event-study and historical-analogy method contracts.
- Product operating model, acceptance protocol, evidence-package ADR and public
maturity roadmap.
- Visible-vintage fingerprint tests and memoisation.
- Fail-closed rule-planner routing for unsupported research families.
- Family/risk/severity-aware evaluation reports with refusal cases and a
separate silent-critical-failure count.
- Public adapter PIT conformance suite and `qf adapter-check`.
- Optional disposable-process execution with wall/CPU/memory containment and
explicit non-sandbox semantics.

### Changed
- Successful report runs emit an evidence package beside the HTML report.
- The quality audit requires the evidence package to verify before awarding
full diagnosability credit.
- Cache input identity now hashes only the data visible at the knowledge date.

## [0.2.0] — 2026-08-12

### Added
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ authors:
website: "https://github.com/leoncuhk"
repository-code: "https://github.com/leoncuhk/quantifact"
license: Apache-2.0
version: 0.2.0
date-released: 2026-08-12
version: 0.3.0a1
date-released: 2026-08-13
keywords:
- llm agents
- investment research
Expand Down
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-3776AB.svg)](pyproject.toml)
[![License Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-6B7280.svg)](LICENSE)
[![Explore a run](https://img.shields.io/badge/explore-a_live_run-14B8A6.svg)](https://leoncuhk.github.io/quantifact/)
[![Status: alpha](https://img.shields.io/badge/status-alpha-f59e0b.svg)](#project-status)

Quantifact is an open-source investment-research system built as an evidence
compiler. Four bounded subsystems turn an ambiguous question into a typed
Expand Down Expand Up @@ -42,6 +43,12 @@ A successful run produces more than a chart:
timings, findings, and output lineage;
- content-addressed results, so changing one task recomputes only what changed.

The durable output is a versioned **Research Evidence Package**, not the chat
or HTML report. It binds each permitted claim to its plan, source vintages and
licences, generated code, materialised-value fingerprint and verdicts, then
protects the package with an integrity hash. Admission means only “fit for
expert review”; it never means investment approval or proof that a claim is true.

If a required check fails, the run stops or repairs the named task. It does not
quietly turn an unverified number into a polished report.

Expand All @@ -53,12 +60,14 @@ or market-data licence required.
```bash
uv add git+https://github.com/leoncuhk/quantifact
qf ask --receipt .qf/run.json
qf verify .qf/report.evidence.json
qf ask --execution process --task-timeout 10 # crash/timeout containment, not a sandbox
```

```text
as_of 2026-08-01 (nothing published later was read)
plan 16 tasks in 5 layers
contracts 68/68 verdicts passed
contracts 70/70 verdicts passed
report .qf/report.html
receipt .qf/run.json
```
Expand Down Expand Up @@ -101,7 +110,17 @@ Quantifact makes those failure modes explicit:
| Expensive iteration | Content-addressed caching recomputes only the affected subgraph |
| Unsafe learning | A lesson must reproduce a failure, fix it, and pass regression before acceptance |

## Four bounded subsystems
## Product value and buyer

Quantifact is intended for research organisations where analyst time, review
cost, data semantics and silent error are expensive. Analysts use it; portfolio
managers challenge its evidence; Heads of Research/CIOs buy research capacity;
risk and compliance buy reconstruction and controls; data leaders buy a common
execution layer. The measurable objective is shorter time to reviewable evidence,
lower reviewer effort, fewer escaped critical errors and more reusable workflows—not
more generated reports. See [the product and operating model](docs/product-operating-model.md).

## Four subsystems that contain investment-research error

The public PAT presentation is best understood as four cooperating subsystems,
not four Python modules. Quantifact implements the same separation because each
Expand Down Expand Up @@ -206,6 +225,13 @@ controls, cache, receipts, and packaged examples are executable and tested.
Production isolation, broad expert evaluation, service reliability, and user
outcomes still require operating evidence.

The current repository-only audit is **41.5/100 — concept prototype**. This is
not a popularity score: it deliberately assigns zero to outcomes a repository
cannot prove. Compiler, PIT, contracts, evidence packages and reproducibility
are executable; expert-held-out accuracy, licensed-data breadth, isolation,
service SLOs and adoption remain open gates. See the [maturity matrix and
roadmap](ROADMAP.md).

[`qf audit`](docs/quality-model.md) makes that boundary measurable and the
release workflow fails closed when required evidence is absent. See
[Production guidance](docs/guides/production.md) before connecting an untrusted
Expand Down Expand Up @@ -236,6 +262,9 @@ one of the most valuable contributions this project can receive.
- [Architecture decisions](docs/adr/)
- [Interactive run explorer](https://leoncuhk.github.io/quantifact/)
- [Quality model and delivery gates](docs/quality-model.md)
- [Product and operating model](docs/product-operating-model.md)
- [Highest-quality acceptance protocol](docs/acceptance.md)
- [Maturity matrix and roadmap](ROADMAP.md)

## Acknowledgements

Expand Down
Loading
Loading