Research: RAG beside MemNet (graph retrieve-then-generate vs HostSearchBridge)

[chouswei]

Kind: research (not a ship request). No rag_query on memnet-mcp. No embeddings in the engine until this issue concludes otherwise.

Doctrine already on the tree: MN-REQ-00 — MemNet is working memory, not the search corpus. Design nest HostSearchBridge (ImportGuard-shaped, outside MemNetSystem): docs/grammar/memnet-host-search-nest.md · draft #75.

External references

1. Adding RAG to your GraphQL API — Adam Cowley, Neo4j Developer Blog (28 Apr 2024). Companion repo: adam-cowley/neo4j-graphql-genai. Note 1 (issue body): retrieve-then-generate on the GraphQL API.
2. RAGFlow (InfiniFlow) — open-source RAG + agent context engine. Note 2: sibling MCP ragflow_retrieval returns chunks.
3. 10 best RAG tools and platforms (2026) — Meilisearch roundup. Note 3: taxonomy of the retrieve hop (search engines, orchestrators, vector DBs). Not a MemNet feature list.
4. HiGram (Yue et al., arXiv 2608.05095, Aug 2026) + GAM / EvoMemKG / MemORAI + ISO GQL Cor 1:2026. Note 21: hierarchical graph memory is view/depth, not Layer; path-local rewrite, not generate.

1. Neo4j GraphQL RAG

What that post actually does (steal the questions, not the stack):

Their move	What it is
Weaviate-style generate(prompt) on a collection	Retrieved records are stuffed into an LLM prompt
@neo4j/graphql @customResolver on Movie / Actor	LLM runs inside the GraphQL API after the graph query
LangChain PromptTemplate + ChatOpenAI	Generation is a resolver, not a second database
generateReview(stars) vs generic generate(prompt)	Restricted vs open prompt on the same retrieve
GraphQL where then generate	Retrieve (graph) then generate — classic RAG, graph as the retriever

That is RAG as a field on the query API. Useful contrast for MemNet: we already rejected the isomorphic footgun (pin_map.generate(prompt) / rag_query on generic MCP). The post is still a clean worked example of retrieve-then-generate with a graph as the retrieve step (not a vector index).

Further reading on the same site (not this issue’s SSOT): GraphRAG study, constant-cost semantic memory.

MemNet mapping (research questions)

Answer these before any engine/MCP work:

1. Retrieve vs generate vs memory. Their generate is prose out of retrieved nodes. MemNet goldfish is shaped GQL subgraph in. Where (if anywhere) may a host emit generated prose — never on pin_map?
2. Graph as retriever vs vector as retriever. Neo4j where + neighbourhood vs HostRagAdapter (Cursor index / docs MCP / embeddings / RAGFlow / Meilisearch-class hybrid search). When is Path-B ingest + pin_map / leftover #73 BoundedMatchFind enough, and when is a host corpus probe required?
3. Resolver vs nest. Their custom resolver sits on the graph API. Our RagHostHook sits beside MutateGate. RAGFlow MCP / LangChain retriever are sibling tools. Is a Weaviate-style generate ever allowed on MemNet, or is that permanently host-only?
4. Prompt on the wire. Open $prompt on every type (their CanGenerate) vs closed RagDecision (propose / none / skip + locators only). Token budget and MN-REQ-11.13 (no corpus dump).
5. LangChain / GraphQL-as-tool. LangChain can call a GraphQL API as a tool (LLM outside). The Neo4j post puts the LLM inside. RAGFlow MCP and the Meilisearch list are LLM-outside. MemNet teach is GQL pin_map/mutate, not GraphQL. Do not dual-teach GraphQL or LangChain as agent wire (ADR-001).
6. Durable cabinet. If M2.5 AgensGraph later grows vector/FTS, keep it behind sessions (hydrate budget) — not LLM↔store RAG as goldfish. Do not vendor ES/Infinity/RAGFlow/Meilisearch/Pinecone as MemNet cabinet. MongoDB “vectors in the primary DB” is the same temptation.

Out of scope for this issue

• Shipping HostRagAdapter or rag_query.
• Nesting HostSearchBridge / EvidenceCentre / RAGFlow under MemNetSystem.
• Storing chunk bodies or embeddings on NODE properties as the memory surface.
• Treating #75 design docs as implemented.
• Vendor InfiniFlow RAGFlow, Meilisearch, Pinecone, or LangChain in this repo.

Done when

A short research note records steal / reject / defer for each question across all references, with an explicit MUST NOT list for agent teach. No code required to close.

• Neo4j: issue body (note 1)
• RAGFlow: note 2
• Meilisearch list: note 3
• 2026 graph-memory + GQL Cor 1: note 21

[chouswei]

Research note 2 — RAGFlow (InfiniFlow)

Not a ship request. Snapshot: public README + mcp/server/server.py (tools ragflow_retrieval / ragflow_list_datasets / ragflow_list_chats) + docs: what is an Agent Context Engine, quickstart, knowledge compilation, Memory. Product ~v0.26.4; they also ship agent templates, DeepDoc, ES/Infinity, MCP (SSE + streamable-http).

What RAGFlow is

A corpus context engine: ingest (parse → chunk → optional compile → index) → multi-recall (keyword + vector + rerank) → cite chunks → optional chat/agent. They explicitly brand three layers:

RAGFlow layer	Job
Knowledge Core	DeepDoc + template chunking + TreeRAG/GraphRAG-style compilation; Elasticsearch or Infinity holds full text and vectors
Memory	Separate product: embed conversation / agent extraction for later reuse
Tool orchestrator	MCP + “tool retrieval” so the prompt does not dump every tool

MCP ragflow_retrieval takes question (+ optional dataset/document ids) and returns JSON chunks (page_size default 10, max 100, copy says “avoid token limits”). That is a sibling retrieve tool, not a session graph.

Their “knowledge compilation” Graph/Tree/PageIndex is LLM-extracted artefacts from documents (Parser → Chunker → Compiler → Indexer), then indexed for Q&A. It is not a goldfish working-memory graph.

Vs Neo4j GraphQL RAG (note 1)

	Neo4j post	RAGFlow
Retriever	Graph where / neighbourhood	Chunk index (ES/Infinity) + optional compiled KG
Generate	generate(prompt) on the GraphQL type	Chat/agent after retrieve; MCP stops at chunks
LLM placement	Inside the graph API resolver	Outside (HTTP/MCP client) — closer to MemNet host
Payload	Node properties into prompt	Chunk bodies + document metadata

Steal / reject / defer (against #77 questions)

#	Steal	Reject	Defer
1 Retrieve / generate / memory	Keep three layers. RAGFlow Memory ≠ MemNet session. Generated prose lives in host chat, not pin_map.	Do not absorb RAGFlow Memory into parts/common/memnet.	Whether a host may record a short claim on a HIT node (already in HostSearchBridge).
2 Graph vs vector retriever	Host corpus probe = RAGFlow-class (or Cursor index). Mission graph = pin_map / ingest locators / leftover #73.	Do not replace Path-B ingest with DeepDoc chunking.	When compiled KG (their Graph template) is worth a host adapter vs BoundedMatchFind.
3 Resolver vs nest	Sibling MCP is the right seam (ragflow_retrieval ≈ HostRagAdapter). Same transport family we already use (streamable-http).	MUST NOT add ragflow_retrieval (or any chunk retrieve) to memnet-mcp.	Official adapter recipe in user-pack only.
4 Prompt / budget	Steal hard page_size and “avoid token limits” honesty. Prefer their document ids/names/location as locators.	Returning full chunks[] as goldfish (MN-REQ-11.13). Open question on MemNet types.	Map RAGFlow chunk → path=/document_id + optional ≤120-char claim.
5 LLM outside vs inside	RAGFlow MCP = LLM calls retrieve (LangChain-as-tool grain). Aligns with reject of Neo4j-in-resolver.	Dual-teach GraphQL or RAGFlow schema as agent wire (ADR-001).	—
6 Durable cabinet	Doc engine (ES/Infinity) stays behind the host, like AgensGraph behind sessions.	Vendor RAGFlow or Infinity as MemNet cabinet. Dual-write session graph ↔ chunk index.	If M2.5 ever grows FTS, still hydrate-budgeted, not MCP retrieve.

Extra contrasts (new)

RAGFlow	MemNet
“Quality in, quality out” + human-edit chunks	MN-REQ-10.7 verify via parser / grep; locators not chunk editors
Knowledge compilation Graph from docs	NODE|EDGE from mission atomise + Path-B locators
Agent Memory (embed history)	Session TTL + pin_map goldfish; chat is never SSOT
Context engine is the product	MN-REQ-00: MemNet sits between pipelines and that engine
16 GB RAM / Docker / embeddings	In-process graph; no embedding models in engine

Updated MUST NOT (agent teach)

• Do not treat RAGFlow (or its MCP) as MemNet.
• Do not paste ragflow_retrieval chunk JSON into add/update properties.
• Do not nest DeepDoc / Infinity / RAGFlow Memory under MemNetSystem.
• Do not teach RAGFlow Graph compilation as GQL wire.

Host recipe (if someone runs both)

1. pin_map (MemNet).
2. Optional: MCP ragflow (separate server) ragflow_retrieval — cap page_size.
3. Atomise locators (document_id, name, location) via MutateGate; discard chunk text.
4. Re-pin_map. Skip entire step 2 when grep/ingest suffices (Path A).

Done-when for #77 still needs a compact steal/reject table covering both Neo4j and RAGFlow; this comment is the RAGFlow half.

[chouswei]

Research note 3 — 10 best RAG tools and platforms (2026) (Meilisearch / Maya Shin; updated 4 Aug 2026)

Not a ship request. Marketing roundup, useful as a taxonomy of the retrieve hop, not as a MemNet feature list. Their own FAQ: RAG tools index data, find relevant hits (vector / keyword / hybrid), and pass that context to the LLM. That sentence is MN-REQ-00’s search corpus. MemNet is the layer between pipeline turns, not that pass-through.

Their ten, grouped

Bucket	Names in the post	Job
Hybrid search engines	Meilisearch, Vespa	FTS + vectors, ranking, typo-tolerance, APIs
Orchestrators	LangChain, Haystack, LlamaIndex, Embedchain	Load → index → retrieve → generate / agents
Retriever quality	RAGatouille (ColBERT late interaction)	Rerank / token-level scoring
Chat-over-docs UI	Verba (Weaviate)	Upload, chunk, highlight sources
Vectors in a database	MongoDB Atlas Vector Search, Pinecone	Embeddings next to app data, or dedicated vector SaaS

Related on the same site (not expanded here): 13 RAG chunking strategies.

Vs notes 1–2

	Neo4j GraphQL	RAGFlow	This list
Product	Graph API + generate resolver	Full context engine + sibling MCP	Shopping list for retrieve (and glue)
LLM	Inside resolver	Outside, calling retrieve	LangChain/LlamaIndex as outside glue
Payload	Node props → prompt	Chunk JSON	Hits → prompt (explicit)

None of the ten is a session goldfish graph. LangChain “memory” in the post is conversation/prompt state, not MemNet pin_map.

Steal / reject / defer (questions 1–6)

#	Steal	Reject	Defer
1	Their FAQ splits index / retrieve / pass-to-LLM. MemNet is none of those three; host chat owns generate.	Do not rebrand MemNet as a “RAG tool” on this list.	—
2	Hybrid (BM25 + vector) is a HostRagAdapter quality knob (Meilisearch/Vespa/Pinecone/RAGFlow ES). Cursor index already fills “good enough” for code.	Do not pick a vector DB inside parts/common/memnet. Path-B ingest ≠ chunking-strategy menu.	Host may choose Meilisearch vs RAGFlow vs Cursor; MemNet does not.
3	Orchestrators (LangChain, Haystack) sit beside MemNet, same as RAGFlow MCP.	LangChain-in-engine; pin_map as a LangChain retriever class in this repo.	User-pack snippet: LangChain tool → locators → add.
4	Ranking, top_k, hybrid weights = SoftHitBudget cousins. Verba’s visible sources ≈ locator emit.	Passing hit text into goldfish.	Map search id + URL/path only.
5	The list assumes LangChain calls search APIs (LLM outside). Reinforces reject of Neo4j-in-resolver.	Dual-teach LangChain / LlamaIndex as agent wire (ADR-001 stays GQL).	—
6	MongoDB “vectors in the primary DB” is the cabinet temptation. Same warning as AgensGraph+FTS: behind sessions, one sync owner, not goldfish.	Vendor Meilisearch/Pinecone/Vespa as M2.5 cabinet. Dual-write graph ↔ vector index.	If a host already has Meilisearch, it is HostRagAdapter, not DurableBuffer.

Extra MUST NOT

• Do not add Meilisearch (or any of the ten) as a MemNet extra.
• Do not treat “best RAG tool” comparisons as MemNet roadmap.
• Do not implement ColBERT / HNSW / chunkers in the engine.

Host recipe (generic, this list)

pin_map → optional search engine or RAG MCP (cap hits) → atomise locators → MutateGate → re-pin_map. The Meilisearch article is how a host picks the middle box, not a change to MemNet.

Note 2 (RAGFlow): #77 (comment)

[chouswei]

Research note 4 — five questions (problem → principles)

SSOT copy also in docs/grammar/memnet-host-search-nest.md (Problem statement). This is the spine for notes 1–3, not a new product.

1. What problem we want to solve

An LLM turn needs two bounded contexts that must not be fused:

Need	Failure if missing
Mission working memory	Ids / tasks / constraints / verified locators vanish when chat scrolls (goldfish).
Corpus lookup	Invention, or a dump of the manual/repo/feed into the prompt (tokens + hallucination).

MN-REQ-00: save wall-clock and tokens while keeping factual accuracy. Product problem: get from “fact in a haystack” to “copyable id on a pin map” without making MemNet the haystack.

2. Root cause

Retrieve, generate, and remember all “put less text in the prompt” but are different:

Job	Mechanism
Retrieve	Index / hybrid / graph where → hits
Generate	Hits + prompt → prose
Remember	Atomised NODE|EDGE → shaped pin_map

Root cause: treating RAG (retrieve+generate) as memory, or memory as a search engine. Symptom: rag_query on memnet-mcp, chunks on note=, pin_map.generate(prompt).

3. Where we meet it

Coding (Cursor index vs MOD/SYM), Multitask (no corpus import), docs/SCPI/RSS (MN-REQ-11.13), EvidenceCentre librarian vs MutateGate, MCP tool zoo, M2.5 “vectors in the cabinet”.

4. Mechanism that can solve it

Split; compose at the host.

corpus → retrieve (host) → hits → atomise locators → MutateGate → pin_map

MemNet = optional HostSearchBridge (soft locators, hard reuse MutateGate, skip valid).

Contrasts (not core): Neo4j generate on the API (note 1); RAGFlow sibling MCP chunks (note 2); Meilisearch-class hybrid search / LangChain glue (note 3).

5. Principles

Two products · goldfish pin_map · Write = display · pins not dumps · soft then hard · skip is valid · one writer · LLM outside retrieve · verify with grep/parser.

Notes 1–3 remain steal/reject tables against these five.

[chouswei]

Research note 5 — role vs big RAG tools; expected size

The roundups (notes 2–3) show the retrieve/generate market is already crowded. MemNet’s job is the small layer they skip.

Role: mission scratch graph — session id, goldfish pin_map, gated mutate, Multitask handoff. Big RAG finds hits; MemNet remembers verified locators and live TSK/USR. MN-REQ-00: between LLM pipelines and data search. Not a competitor to RAGFlow/Meilisearch/LangChain/Pinecone.

Expected size (stay small):

Axis	Expectation
Product	Engine + generic MCP only
Runtime	In-process first; not a 16 GB RAG stack
Goldfish	~depth 2, 50 rows
Session cap	5000 non-law rows default
Grain	Tens–hundreds of atoms / mission
Time	Session / mission TTL, not a KB

If it becomes the archive, it is no longer MemNet (cabinet or RAG index — downstream).

Grammar: docs/grammar/memnet-host-search-nest.md § Role vs big RAG tools.

[chouswei]

Research note 6 — RAM target: few tens of MiB per session

User expectation: a live MemNet session should run in a few tens of megabytes (order 10–50 MiB for the session graph + pin-map buffers).

That is the size class of a scratch pad, not RAGFlow/ES/embeddings.

Honesty vs shipped Caps:

• Goldfish emit: 50 rows — small.
• Store: 5000 non-law rows, 32 fields, 4096 bytes/value. Worst-case fill would be hundreds of MiB — so row caps do not yet enforce the RAM target.
• Typical atomised missions (short locators, hundreds of nodes) should sit in a few MiB plus the CPython process baseline (also tens of MiB; do not count the interpreter as “the session”).

Implication: keep HostSearchBridge locator-only; no chunk bodies in the graph. Optional later: MEMNET_MAX_SESSION_BYTES (not shipped). Grammar table updated.

[chouswei]

Research note 7 — pin the role (not “the small RAG”)

Pinned role: MemNet is mission working memory, good enough to keep the working set of a few technical documents (structure, locators, distilled atoms; source files stay on disk) and fast enough to re-read that set every turn (pin_map, in-process, ~50 rows).

Tens of MiB is the fit for that job, not a claim that the product is insignificant. RAG already owns the library. MemNet owns the open manuals-and-mission on the bench.

MUST NOT describe MemNet as “the small RAG tool” or as too thin to hold tec-doc work. MUST NOT grow it into the archive to look “big enough”.

Grammar § Role (pinned) + README one-liner.

[chouswei]

Research note 8 — hundreds of MiB still in role

User correction: hundreds of MiB per session is also fine.

Band	Meaning
Few MiB	Typical sparse atoms
Tens of MiB	Comfortable tec-doc working set
Hundreds of MiB	Still MemNet (fatter working set / more CMD/SEC pins)
Gigabytes	Out of role — RAG index / cabinet

Do not treat the 5000×4KiB worst case as a defect. Prefer atomise so goldfish stays fast; do not grow to GB to look like RAG. Grammar/README/CapsPolicy note updated.

[chouswei]

Research note 9 — we face the RAG-shaped issue inside the session

Yes. Same shape (relevant subset into a small context window), different haystack.

Haystack	Who	Mechanism
World's corpus / PDFs	Host RAG	Chunks, hybrid search
Live session (few tec docs, up to hundreds of MiB)	MemNet	pin_map / walk / leftover #73 find

Goldfish exists because the working set is already too big to dump. Do not solve in-session slice with a second vector store. Do not pretend MemNet never has a retrieve problem.

Grammar § Role updated.

[chouswei]

Research note 10 — mathematical principles

Search, not a ship list. Grammar table: docs/grammar/memnet-host-search-nest.md section Mathematical principles.

1. Information bottleneck (shared objective)

Tishby, Pereira, Bialek 1999. Applied to RAG noise: Zhu et al., An Information Bottleneck Perspective for Effective Noise Filtering on RAG (ACL 2024); arXiv 2406.01549.

Minimise I(X-tilde; X | Q) (conciseness) minus beta times I(X-tilde; Y | Q) (correctness). Their Fig. 1B: if retrieve does not help, optimal compression is empty — Path A skip.

Related: MemFly (on-the-fly memory via IB). Steal the objective; reject putting their LLM merger in the engine.

MemNet: X = session graph, Q = anchor, X-tilde = pin_map, Y = this turn’s accurate act. Host RAG: X = corpus.

2. Rate–distortion

Shannon: spend bits only to cut task error. MN-REQ-00 is this in product language (tokens / wall-clock vs accuracy). Session RAM band (tens–hundreds of MiB) is a rate cap on session X, not on the library.

3. Why ego-hop (intractability)

Optimal evidence subgraph is NP-hard (Johnson and Garey). GRAG (Hu et al.): retrieve k-hop ego-graphs then prune — search space O(|V|) egos, not 2^(|V|+|E|).

That is already pin_map(anchor, depth) (RSV uses the same expand). TopoRAG diameter-constrained NN: relevance and small diameter — our depth. COSMOS submodular maximisation under a budget — our max_rows; greedy approximation if we ever rank; not required to ship.

4. Associative memory (why the LLM still needs a slice)

Ramsauer et al., Hopfield Networks is All You Need: transformer attention equals one modern-Hopfield retrieve step. The model already does content-addressable retrieve inside the prompt. Vector RAG is long-term associative memory; attention is short-term.

MemNet should not reimplement Hopfield. It should feed a low-rate structured cue (shaped GQL) so in-prompt attention has something accurate to complete. Chunk dumps spend rate on the wrong patterns.

Steal / reject

Steal: IB as the name of goldfish; ego k-hop as the tractable subgraph; empty retrieve when skip is valid; keep the prompt small and structured.

Reject: train a neural IB filter in memnet-llm; embed session nodes and ANN-search as default pin_map; soft graph tokens / GNN prompts in the engine; claim MemNet is a Hopfield net.

Two haystacks, one Lagrangian. Do not spend beta on corpus X inside the session, or on session X inside host RAG.

[chouswei]

Research note 11 — more math (hardness, packing, diameter)

Supplement to note 10. Still search, not a ship list. Grammar table updated on cursor/host-search-nest-3a36.

Hardness of picking the evidence subgraph

• GRAG (arXiv 2405.16506): exhaustive textual-subgraph search is NP-hard (Johnson and Garey). Tractable surrogate = index k-hop ego-graphs, retrieve, then prune. That is already pin_map(anchor, depth).
• BubbleRAG (arXiv 2603.20309): Optimal Informative Subgraph Retrieval reduces to Group Steiner Tree — NP-hard and APX-hard. Heuristic bubble expansion, not an exact solver.
• AGRAG (arXiv 2511.05549): Minimum Cost Maximum Influence subgraph is NP-hard; greedy 1-hop expansion.
• Topology of Reasoning (arXiv 2602.19240): connected subcomplex as Prize-Collecting Steiner Tree on cell complexes.

Steal: ego k-hop from a known id. Reject: GST / PCST / MCMI solvers in memnet-llm. Leftover #73 is bounded find, not Steiner.

Rate, packing, greedy guarantees

• Shannon rate-distortion (1959) + Tishby IB (1999): spend bits only to cut task error. MN-REQ-00 in product language.
• Directed information gamma-covering (arXiv 2510.00079): query-agnostic packing with set-cover / budgeted-coverage greedy (Johnson 1974; Nemhauser, Wolsey, Fisher 1978).
• Nemhauser et al. 1978: monotone submodular max under cardinality — greedy is (1-1/e)-approx. RAG packers instantiate this: AdaGReS (arXiv 2512.25052), PACMS (arXiv 2606.20047), What survives into context (arXiv 2607.00725).

Steal: max_rows / fanout are the budget; goldfish exists so the answer can survive into the prompt. Reject: learned packer / MMR / ColBERT in the engine.

Diameter and cohesion (without cosine)

• TopoRAG (ACL Findings 2026): diameter-constrained ANN. MemNet already has diameter via depth; metric = edge hops, not embedding ANN.
• DA-RAG (arXiv 2602.08545): k-truss community search is NP-hard; connected k-truss has a diameter bound. Reject k-truss in the engine.

IB / attention (note 10 recap + proxies)

• Zhu et al. ACL 2024 (arXiv 2406.01549): empty compression when retrieve is useless = Path A skip.
• QUITO-X (arXiv 2408.10497): cross-attention as a proxy for mutual information — inside the LLM, not MutateGate.
• Ramsauer et al. (arXiv 2008.02217): attention = one modern-Hopfield step. MemNet feeds the cue; it is not the Hopfield net.

Steal / reject (this note)

Steal	Reject
Name goldfish as session IB / rate cap	Train beta-IB / neural noise filter in memnet-llm
Ego k-hop as the polynomial subgraph	GST, PCST, MCMI, k-truss peelers
Cardinality budget = max_rows	Learned submodular packer / ANN on session nodes
depth as diameter	Cosine diameter / HNSW in-process
Empty retrieve when skip is valid	Claim MemNet is Hopfield or GraphRAG

Two haystacks, one Lagrangian still holds.