SignalProof

Contract-driven telemetry and detection validation for ATT&CK-mapped security testing.

SignalProof verifies whether expected security events, required fields, ingestion paths, and detection alerts exist after a controlled ATT&CK-mapped test.

Why SignalProof exists

Security teams frequently write detection rules under the assumption that telemetry is present and correctly normalized. In practice, agents may be unhealthy, required fields may be missing, ingestion may be delayed, and rules may never fire.

SignalProof turns those assumptions into explicit, testable telemetry contracts.

Core capabilities

• YAML telemetry contracts
• Event and field-level assertions
• Ingestion and detection latency validation
• Wazuh/OpenSearch alert collection
• Blind-spot classification
• Evidence reports with SHA-256 manifests
• PostgreSQL validation history
• Regression and recovery comparison
• FastAPI REST API
• Streamlit dashboard
• Automated tests and GitHub Actions CI

Architecture

SignalProof separates collection, validation, diagnosis, persistence, and presentation so that additional SIEM collectors can be added without changing the core validation engine.

Telemetry contracts

A contract defines what must exist for a technique to be considered detection-ready:

contract:
  id: windows-powershell-t1059-001
  version: "1.0"

attack:
  technique_id: T1059.001
  technique_name: PowerShell

events:
  - assertion_id: sysmon-process-create
    source: sysmon
    event_id: 1
    required: true
    required_fields:
      - win.eventdata.image
      - win.eventdata.commandLine
      - win.eventdata.processGuid
    maximum_ingestion_latency_seconds: 30

detection:
  required: true
  backend: wazuh
  rule_id: "100201"
  minimum_alert_level: 8
  maximum_detection_latency_seconds: 60

Validation outcomes

SignalProof marks a run as:

• passed
• degraded
• failed

It also produces an evidence-based diagnosis such as:

• telemetry_unobserved
• collection_failure
• forwarding_failure
• parsing_failure
• ingestion_latency_failure
• detection_failure
• detection_quality_failure
• detection_latency_failure
• workflow_failure
• multiple_failures

Regression testing

SignalProof compares persisted runs to detect regressions and recovery.

signalproof compare-runs `
  --previous "SP-DEMO-BASELINE-001" `
  --current "SP-DEMO-DETECTION-MISSING-001"

Wazuh integration

The Wazuh collector queries alert evidence through the Wazuh Indexer API using a read-only workflow.

Current live-lab mode:

• wazuh-alerts-*: supported
• wazuh-archives-*: optional and not enabled in the shared lab
• live mode: detection validation
• fixture mode: full event and field validation

Evidence reports

Each validation run can generate:

• report.json
• report.md
• manifest.json
• SHA-256 hashes for generated and source evidence

Sanitized examples are available in reports/samples.

REST API

SignalProof exposes contract validation, fixture execution, Wazuh health, history, and regression endpoints through FastAPI.

Start it with:

uvicorn signalproof.api.app:app --reload

Then open:

http://127.0.0.1:8000/docs

Quick start

Requirements

• Python 3.12 or 3.13
• Docker
• PostgreSQL through Docker Compose
• Optional Wazuh lab for live alert validation

Install

git clone https://github.com/YOUR-USERNAME/signalproof.git
cd signalproof

python -m venv .venv
.\.venv\Scripts\Activate.ps1

pip install -e ".[dev]"

Start PostgreSQL

docker compose -f docker-compose.phase9.yml up -d

Configure the database

$env:SIGNALPROOF_DATABASE_URL = `
  "postgresql+psycopg://signalproof:signalproof@127.0.0.1:5433/signalproof"

Initialize tables

signalproof init-db

Seed demonstration runs

.\scripts\seed_demo_runs.ps1

Launch the dashboard

streamlit run signalproof/dashboard/app.py

Testing

python -m compileall signalproof
ruff check signalproof tests
pytest -v

Safety model

SignalProof is designed for authorized defensive validation.

• It does not execute attack techniques.
• Live collection is read-only.
• Controlled tests must be executed separately in an isolated lab.
• Credentials are supplied through environment variables.
• .env files and private evidence must not be committed.
• Duplicate run IDs are rejected to protect evidence history.

Limitations

• The current Wazuh lab exposes alert indices but not raw archive indices.
• Live raw-event normalization is not implemented.
• Authentication and RBAC are outside the current MVP.
• Streamlit is used as an engineering dashboard rather than a production frontend.

Roadmap

• Wazuh archive normalization
• Elastic and Splunk collectors
• Scheduled validation suites
• Multi-technique campaigns
• Authentication and RBAC
• Web-based telemetry contract editor

License

MIT