Skip to content

Extend datacortex from retrieval-only to predictive embeddings #6

Description

@plur9

Context

Datacortex stores semantic embeddings for similarity search. Kumo's relational pretraining produces predictive embeddings — vectors learned from outcomes, not from text similarity. Same storage substrate, different learning objective.

Reference: https://kumo.ai/company/news/low-latency-high-throughput-predictions-with-kumorfm-2-fine-tuning/

Problem

Today datacortex answers "what's similar to this?" but not "what's about to happen to this?" — even though the data to answer the second question (outcomes: task completion, contact response, venture hypothesis confirmation) is sitting in Datacore.

Proposal

Add a second embedding kind to the datacortex store: outcome-supervised vectors trained on labeled events.

Examples:

  • Task embedding trained to predict "was this completed within deadline?"
  • Contact embedding trained to predict "will this contact respond in 30 days?"
  • Venture-hypothesis embedding trained to predict "did this hypothesis hold?"

Reuse the existing datacortex vector backend; add a model field per embedding. Retrieval can still query by similarity, but ranked by predicted outcome.

Acceptance

  • Datacortex schema accepts multiple embedding kinds per entity
  • At least one outcome-supervised embedding shipped (recommend: task completion likelihood)
  • Datacortex queries can rank by predicted outcome, not just similarity
  • Documented in datacortex module README

Depends on

  • "Relational router model" issue (model providing the outcome predictions)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority:lowCosmetic, nice-to-have, minor improvementsstatus:deferredDeferred — criteria not met yet, revisit when threshold is reached

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions