Add and rank usage-based navigation hints by prompt similarity

README.md

@@ -22,4 +22,4 @@ All solutions within this repository are provided under the [Apache 2.0](https:/
 
 ## Disclaimer
 
-This repository and its contents are not an official Google product.
+This repository and its contents are not an official Google product.

okf/SPEC.md

@@ -116,6 +116,7 @@ sense for the knowledge being captured.
 path/to/bundle/
   index.md                      # Optional. Directory listing for progressive disclosure.
   log.md                        # Optional. Chronological history of updates.
+  usage.yaml                    # Optional. Advisory usage/navigation hints.
   <concept>.md                  # A concept at the bundle root.
   <subdirectory>/               # Subdirectories organize concepts into groups.
     index.md
@@ -133,13 +134,14 @@ A bundle MAY be distributed as:
 
 ### 3.1 Reserved filenames
 
-The following filenames have defined meaning at any level of the
-hierarchy and MUST NOT be used for concept documents:
+The following filenames have defined meaning and MUST NOT be used for
+concept documents:
 
 | Filename   | Purpose                          |
 |------------|----------------------------------|
 | `index.md` | Directory listing. See §8.       |
 | `log.md`   | Update history. See §9.          |
+| `usage.yaml` | Bundle-root usage/navigation hints. See §9.1. |
 
 All other `.md` files are concept documents.
 
@@ -518,9 +520,7 @@ sections, each grouping concepts under a heading:
 
 * [Title 1](relative-url-1) - short description of item 1
 * [Title 2](relative-url-2) - short description of item 2
-
 # Another Section
-
 * [Subdirectory](subdir/) - short description of the subdirectory
 ```
 
@@ -551,6 +551,47 @@ Date headings MUST use ISO 8601 `YYYY-MM-DD` form. Log entries are prose;
 the leading bold word (`**Update**`, `**Creation**`, `**Deprecation**`) is
 a convention, not a requirement.
 
+### 9.1 Usage and navigation hints
+
+A bundle MAY contain a `usage.yaml` file at its root. It records aggregated,
+derived observations about which concepts have historically been useful for
+an intent and set of conditions. It is a navigation signal, not knowledge:
+consumers MUST treat it as advisory and MUST read the referenced concept as
+the source of truth.
+
+The initial schema is:
+
+```yaml
+version: 1
+usage_hints:
+  - concept: metrics/revenue.md
+    intent: revenue_calculation
+    conditions:
+      period: monthly
+      aggregation: total
+    access_count: 514
+    successful_count: 462
+    last_accessed: 2026-08-20T08:30:00Z
+```
+
+`concept` is a bundle-relative path to a non-reserved Markdown concept.
+`intent` and condition keys are producer-defined strings. `access_count` is
+the number of observed accesses; `successful_count`, when present, counts
+accesses that contributed to a successful resolution and MUST NOT exceed
+`access_count`. `last_accessed` is an optional ISO 8601 timestamp.
+
+Usage data SHOULD be aggregated across consumers and MAY be updated in
+batches. Producers MUST NOT store raw user queries or per-user history in
+this file, and implementations MUST NOT require personally identifying
+information in `usage.yaml`. Consumers MAY use recency, success rate,
+frequency, keyword search, vector search, or graph traversal when ranking
+candidates; OKF does not prescribe a matching algorithm. A usage hint MUST
+NOT be used to answer a request without retrieving and evaluating the
+referenced concept.
+
+The file is optional. Consumers that do not support it MUST ignore it, and
+consumers MUST tolerate unknown top-level and entry fields.
+
 ---
 
 ## 10. Attested computations concept
@@ -740,8 +781,8 @@ A bundle is **conformant** with OKF v0.2 if:
 1. Every non-reserved `.md` file in the tree contains a parseable YAML
    frontmatter block.
 2. Every frontmatter block contains a non-empty `type` field.
-3. Every reserved filename (`index.md`, `log.md`) follows the structure in
-   §8 and §9 respectively when present.
+3. Every reserved filename (`index.md`, `log.md`, `usage.yaml`) follows the
+  structure in §8, §9, and §9.1 respectively when present.
 
 When the trust, lifecycle, provenance, or computation families are
 present, producers SHOULD follow §5 through §10, and consumers:

okf/examples/usage-hints/index.md

@@ -0,0 +1,8 @@
+# Usage Hints Example
+
+This bundle demonstrates optional advisory navigation metadata. The metric
+documents remain the source of truth; [usage.yaml](usage.yaml) only suggests
+where a consumer might look first.
+
+* [Revenue](metrics/revenue.md)
+* [Regional Revenue](metrics/regional-revenue.md)

okf/examples/usage-hints/metrics/regional-revenue.md

@@ -0,0 +1,9 @@
+---
+type: Metric
+title: Regional Revenue
+description: Recognized revenue grouped by region.
+---
+
+# Definition
+
+Recognized revenue is grouped by the order's region.

okf/examples/usage-hints/metrics/revenue.md

@@ -0,0 +1,10 @@
+---
+type: Metric
+title: Revenue
+description: Recognized revenue for a requested period.
+---
+
+# Definition
+
+Recognized revenue is the total value of completed orders in the requested
+period.

okf/examples/usage-hints/usage.yaml

@@ -0,0 +1,18 @@
+version: 1
+usage_hints:
+  - concept: metrics/revenue.md
+    intent: revenue_calculation
+    conditions:
+      period: monthly
+      aggregation: total
+    access_count: 514
+    successful_count: 462
+    last_accessed: 2026-08-20T08:30:00Z
+  - concept: metrics/regional-revenue.md
+    intent: revenue_calculation
+    conditions:
+      group_by: region
+      aggregation: total
+    access_count: 287
+    successful_count: 241
+    last_accessed: 2026-08-19T16:42:11Z

okf/src/reference_agent/bundle/__init__.py

@@ -1,11 +1,27 @@
 from reference_agent.bundle.document import OKFDocument, REQUIRED_FRONTMATTER_KEYS
 from reference_agent.bundle.index import regenerate_indexes
 from reference_agent.bundle.paths import concept_id_to_path, path_to_concept_id
+from reference_agent.bundle.usage import (
+    UsageError,
+    UsageFile,
+    UsageHint,
+    load_usage,
+    rank_usage,
+    record_usage,
+    save_usage,
+)
 
 __all__ = [
     "OKFDocument",
     "REQUIRED_FRONTMATTER_KEYS",
     "concept_id_to_path",
     "path_to_concept_id",
     "regenerate_indexes",
+    "UsageError",
+    "UsageFile",
+    "UsageHint",
+    "load_usage",
+    "rank_usage",
+    "record_usage",
+    "save_usage",
 ]

okf/src/reference_agent/bundle/usage.py

@@ -0,0 +1,214 @@
+from __future__ import annotations
+
+import os
+import tempfile
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path, PurePosixPath
+from typing import Any
+
+import yaml
+
+USAGE_FILENAME = "usage.yaml"
+USAGE_VERSION = 1
+
+
+
+class UsageError(ValueError):
+    """Raised when usage metadata is malformed or unsafe."""
+
+
+@dataclass
+class UsageHint:
+    concept: str
+    intent: str
+    conditions: dict[str, Any] = field(default_factory=dict)
+    access_count: int = 0
+    successful_count: int | None = None
+    last_accessed: str | None = None
+    extensions: dict[str, Any] = field(default_factory=dict)
+
+    @classmethod
+    def from_mapping(cls, value: Any) -> "UsageHint":
+        if not isinstance(value, dict):
+            raise UsageError("Each usage_hints entry must be a mapping")
+        concept = value.get("concept")
+        intent = value.get("intent")
+        conditions = value.get("conditions", {})
+        if not isinstance(concept, str) or not concept:
+            raise UsageError("Usage hint concept must be a non-empty string")
+        if not isinstance(intent, str) or not intent:
+            raise UsageError("Usage hint intent must be a non-empty string")
+        if not isinstance(conditions, dict):
+            raise UsageError("Usage hint conditions must be a mapping")
+        access_count = _non_negative_int(value.get("access_count", 0), "access_count")
+        successful_raw = value.get("successful_count")
+        successful_count = (
+            None
+            if successful_raw is None
+            else _non_negative_int(successful_raw, "successful_count")
+        )
+        if successful_count is not None and successful_count > access_count:
+            raise UsageError("successful_count cannot exceed access_count")
+        last_accessed = value.get("last_accessed")
+        if last_accessed is not None and not isinstance(last_accessed, str):
+            raise UsageError("last_accessed must be an ISO-8601 string")
+        extensions = {
+            key: item
+            for key, item in value.items()
+            if key not in {
+                "concept", "intent", "conditions", "access_count",
+                "successful_count", "last_accessed",
+            }
+        }
+        return cls(
+            concept=concept,
+            intent=intent,
+            conditions=dict(conditions),
+            access_count=access_count,
+            successful_count=successful_count,
+            last_accessed=last_accessed,
+            extensions=extensions,
+        )
+
+    def to_mapping(self) -> dict[str, Any]:
+        value: dict[str, Any] = {
+            "concept": self.concept,
+            "intent": self.intent,
+        }
+        if self.conditions:
+            value["conditions"] = self.conditions
+        value["access_count"] = self.access_count
+        if self.successful_count is not None:
+            value["successful_count"] = self.successful_count
+        if self.last_accessed is not None:
+            value["last_accessed"] = self.last_accessed
+        value.update(self.extensions)
+        return value
+
+
+@dataclass
+class UsageFile:
+    hints: list[UsageHint] = field(default_factory=list)
+    version: int = USAGE_VERSION
+    extensions: dict[str, Any] = field(default_factory=dict)
+
+    @classmethod
+    def from_mapping(cls, value: Any) -> "UsageFile":
+        if not isinstance(value, dict):
+            raise UsageError("usage.yaml must contain a mapping")
+        version = value.get("version", USAGE_VERSION)
+        if version != USAGE_VERSION:
+            raise UsageError(f"Unsupported usage.yaml version: {version!r}")
+        raw_hints = value.get("usage_hints", [])
+        if not isinstance(raw_hints, list):
+            raise UsageError("usage_hints must be a list")
+        hints = [UsageHint.from_mapping(item) for item in raw_hints]
+        extensions = {
+            key: item for key, item in value.items() if key not in {"version", "usage_hints"}
+        }
+        return cls(hints=hints, version=version, extensions=extensions)
+
+    def to_mapping(self) -> dict[str, Any]:
+        value = {
+            "version": self.version,
+            "usage_hints": [hint.to_mapping() for hint in self.hints],
+        }
+        value.update(self.extensions)
+        return value
+
+
+def _non_negative_int(value: Any, name: str) -> int:
+    if isinstance(value, bool) or not isinstance(value, int) or value < 0:
+        raise UsageError(f"{name} must be a non-negative integer")
+    return value
+
+
+def _validate_concept_path(concept: str) -> None:
+    path = PurePosixPath(concept)
+    if path.is_absolute() or ".." in path.parts or path.suffix != ".md":
+        raise UsageError(f"Invalid usage hint concept path: {concept!r}")
+    if path.name in {"index.md", "log.md", "usage.yaml"}:
+        raise UsageError(f"Usage hint cannot target reserved file: {concept!r}")
+
+
+def load_usage(bundle_root: Path) -> UsageFile:
+    path = bundle_root / USAGE_FILENAME
+    if not path.exists():
+        return UsageFile()
+    try:
+        raw = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+    except yaml.YAMLError as error:
+        raise UsageError(f"Invalid YAML in {USAGE_FILENAME}: {error}") from error
+    usage = UsageFile.from_mapping(raw)
+    for hint in usage.hints:
+        _validate_concept_path(hint.concept)
+    return usage
+
+
+def save_usage(bundle_root: Path, usage: UsageFile) -> Path:
+    bundle_root.mkdir(parents=True, exist_ok=True)
+    path = bundle_root / USAGE_FILENAME
+    payload = yaml.safe_dump(usage.to_mapping(), sort_keys=False)
+    fd, temporary = tempfile.mkstemp(prefix=".usage-", dir=bundle_root, text=True)
+    try:
+        with os.fdopen(fd, "w", encoding="utf-8") as stream:
+            stream.write(payload)
+        os.replace(temporary, path)
+    finally:
+        if os.path.exists(temporary):
+            os.unlink(temporary)
+    return path
+
+
+def record_usage(
+    bundle_root: Path,
+    concept: str,
+    intent: str,
+    conditions: dict[str, Any] | None = None,
+    *,
+    successful: bool = False,
+    accessed_at: datetime | None = None,
+) -> UsageHint:
+    _validate_concept_path(concept)
+    if not intent:
+        raise UsageError("intent must be a non-empty string")
+    normalized_conditions = dict(conditions or {})
+    usage = load_usage(bundle_root)
+    for hint in usage.hints:
+        if (hint.concept, hint.intent, hint.conditions) == (
+            concept,
+            intent,
+            normalized_conditions,
+        ):
+            break
+    else:
+        hint = UsageHint(concept=concept, intent=intent, conditions=normalized_conditions)
+        usage.hints.append(hint)
+    hint.access_count += 1
+    if successful:
+        hint.successful_count = (hint.successful_count or 0) + 1
+    hint.last_accessed = (accessed_at or datetime.now(timezone.utc)).isoformat()
+    save_usage(bundle_root, usage)
+    return hint
+
+
+def rank_usage(
+    bundle_root: Path,
+    intent: str,
+    conditions: dict[str, Any] | None = None,
+) -> list[UsageHint]:
+    requested = dict(conditions or {})
+    candidates = [
+        hint
+        for hint in load_usage(bundle_root).hints
+        if not intent or hint.intent == intent
+    ]
+    def score(hint: UsageHint) -> tuple[int, int, str]:
+        matches = sum(
+            hint.conditions.get(key) == value
+            for key, value in requested.items()
+        )
+        return matches, hint.access_count, hint.concept
+
+    return sorted(candidates, key=score, reverse=True)