Skip to content

test: add unit + integration test infrastructure for kernel components - #553

Open
RyamL1221 wants to merge 13 commits into
agiresearch:mainfrom
RyamL1221:feature/CI-test-expansion
Open

test: add unit + integration test infrastructure for kernel components#553
RyamL1221 wants to merge 13 commits into
agiresearch:mainfrom
RyamL1221:feature/CI-test-expansion

Conversation

@RyamL1221

Copy link
Copy Markdown
Collaborator

Summary

Adds CI test infrastructure covering components that previously had zero automated coverage, and wires up two new GitHub Actions workflows to run them.

New workflows:

  • test-unit.yml — fast (~2-3 min), no external services, runs on every PR/push to main
  • test-memory-integration.yml — spins up Ollama + ChromaDB, runs real Mem0 provider and cross-agent shared-memory integration tests (~10-15 min)

New test coverage (123 tests):

  • MemoryFormatter — 27 tests, all formatting branches (profile/task_context/conversation, generic fallback, malformed input, exception path)
  • IntentRouter — 35 tests, keyword classification scoring, confidence thresholds, LLM fallback (mocked, no network calls)
  • ConversationExtractor — 12 tests, metadata construction, user_id fallback, error handling
  • FIFOScheduler — 17 tests, _execute_syscall and _execute_batch_syscalls execution paths, including failure handling
  • SyscallExecutor — 21 tests, request dispatch, syscall factory, write-barrier stamping (mocked against real MemoryWriteBarrier via spec=)
  • FastAPI endpoints (launch.py) — 11 tests, request validation, /status, /query, /user/select routing, all with mocked kernel components (no real LLM/memory backend, verified with all provider API keys unset)

New integration test:

  • test_shared_memory_e2e.py — full write → retrieve → inject → format pipeline across two writer agents and one reader agent, against real Ollama + ChromaDB. Skips gracefully (pytest.skip) if Ollama is unreachable, verified via a test-only OLLAMA_SKIP env var that exercises the real reachability-check code path.

Infrastructure:

  • pytest.ini + tests/conftest.py register an integration marker so pytest -m "not integration" / -m integration cleanly separate the two test tiers
  • scripts/patch_config_for_memory_tests.py — small script to configure config.yaml for the mem0/Ollama/ChromaDB test setup, reused by both the workflow and the E2E test
  • README updated with a "Running Tests" section: how to run unit vs. integration tests, per-file skip-vs-fail behavior for integration tests, and a table of known pre-existing issues (see below)

Known pre-existing issues (found, not introduced, by this PR)

These are excluded via --ignore in test-unit.yml and documented in the README so they don't block CI or confuse future contributors:

File Issue
tests/modules/llm/openai/test_concurrent.py Broken import (pre-existing)
tests/modules/llm/openai/test_single.py Broken import (pre-existing)
tests/modules/memory/test_memory.py Broken import (pre-existing)
tests/modules/memory/test_memory_user_identity_isolation.py Order-dependent failure — passes in isolation, fails in full suite due to ConfigManager singleton state pollution from earlier tests

None of these are caused by this PR; they were discovered while wiring up the unit workflow and are called out explicitly rather than silently worked around.

Testing

  • All new unit tests verified locally, passing: 375 total unit tests (252 pre-existing + 123 new), 0 failures
  • test_shared_memory_e2e.py verified locally against real Ollama + ChromaDB (pass) and with OLLAMA_SKIP=1 (skip)
  • All workflow YAML files validated (yaml.safe_load)
  • cancel-workflow.yml confirmed to already cover the two new workflows (workflow_id: all, no update needed)
  • This is the first run of test-unit.yml / test-memory-integration.yml against GitHub's actual runners — opening this PR specifically to validate them in the real CI environment before merge

Documentation

No docs/ changes — this PR adds test infrastructure and coverage only, no new features or behavior changes to document.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant