- 🎯 Building production-grade AI systems — LLM gateways, RAG pipelines, eval harnesses, multi-agent orchestration
- 🏦 Domain focus on Banking & FinTech — credit risk, fraud detection, compliance-aware chatbots
- 🧠 15-project AI Engineering portfolio in progress: type-safe Python, mypy strict, ≥80 % test coverage on every repo
- 🤖 This profile is partially maintained by an AI pipeline — the status line above is written daily by an LLM reading my commit activity, and every animation here is hand-built SVG living in
/assets - ✉️ Open to AI Engineer roles (remote / EU / Turkey) —
ulusoy.metehan03@gmail.com
Tip
✅ Just shipped: Project 06 — Hybrid-Search RAG Pipeline (dense + BM25 + RRF fusion + cross-encoder rerank + citation verification). 🚧 Now building: Project 07 of 15 — semantic caching layer (embedding-similarity response cache + TTL + cache-quality eval). Star the profile to follow new releases.
|
|
Each repo: production-shaped Python, FastAPI / Click CLI / GitHub Action, Pydantic v2, mypy strict, ruff lint, ≥ 80 % pytest coverage, Dockerfile, ADRs, MIT license. No real network calls in tests.
| # | Project | What it does | Stack |
|---|---|---|---|
| 01 | model-regression-detection | CI/CD-style regression gate for LLM features. Async eval runner + LLM-as-judge + 7-run drift, PR comment bot, merge-block on critical regressions. | Python · Pydantic · OpenAI · GitHub Actions |
| 02 | llm-cost-autopilot | Multi-provider LLM gateway. sklearn complexity classifier routes each request to the cheapest acceptable model; async LLM-as-judge sampler verifies quality. >50 % cost reduction vs all-gpt-4o. | FastAPI · scikit-learn · OpenAI · Anthropic · Ollama |
| 03 | failure-forensics | Observability + auto root-cause analysis for multi-step AI pipelines. Decorator-based span tracing, parallel LLM-as-judge backward trace, atomic feedback-to-eval loop, Streamlit explorer. | FastAPI · OpenTelemetry · SQLite WAL · Streamlit |
| 04 | self-healing-docs | GitHub Action that detects when code changes made docs stale, then opens an auto-fix PR or flags affected sections for review. Embedding-based code-to-docs link graph + LLM staleness verifier + style-preservation pass. | GitHub Actions · OpenAI embeddings · PyGithub · unidiff |
| 05 | llm-arbitration | Multi-agent verdict synthesis. Three specialist critics on three different providers grade an output along distinct dimensions; pure disagreement detector + adjudicator agent produce a single confidence-scored verdict. | FastAPI · OpenAI · Anthropic · Ollama · LangGraph-style DAG |
| 06 | rag-hybrid-search | Production-grade RAG pipeline. Multi-format ingestion, dense + BM25 retrieval with reciprocal-rank fusion, cross-encoder rerank, citation verification, and a retrieval-quality eval set. | FastAPI · sentence-transformers · BM25 · Docker |
07–15 in progress: semantic caching, text-to-SQL with guardrails, prompt A/B testing, LoRA fine-tuning, LLM gateway with rate limiting, AI feature flags, eval-set generators, multi-modal document processing, and an agent orchestration platform.
%%{init: {'theme':'base','themeVariables':{'primaryColor':'#1a1b27','primaryTextColor':'#c9d1d9','primaryBorderColor':'#A855F7','lineColor':'#A855F7','secondaryColor':'#24283b','tertiaryColor':'#16161e','fontFamily':'monospace','fontSize':'14px'}}}%%
flowchart LR
D[📄 Docs / Data] --> C[Chunker]
C --> V[(Vector DB)]
Q[🔍 Query] --> H[Hybrid Retriever<br/>dense + BM25 + RRF]
V --> H
H --> X[Cross-encoder<br/>Reranker]
X --> G[🧠 LLM<br/>+ semantic cache]
G --> CV[Citation<br/>Verifier]
CV --> E{Eval Harness<br/>LLM-as-judge}
E -->|pass| O[✅ Ship]
E -->|fail| H
E -.->|drift alert| M[📊 Observability<br/>traces + forensics]
classDef hot fill:#2a1b3d,stroke:#A855F7,stroke-width:2px,color:#e0c7ff
class G,E hot
| Project | What it does | Tech |
|---|---|---|
| credit-risk-scoring | Credit default prediction with SHAP explainability and banking metrics (KS, Gini, PSI). | XGBoost · scikit-learn · SHAP · FastAPI |
| fraud-detection-system | Real-time fraud detection — ML ensemble + rule-based alerts. | XGBoost · Isolation Forest · FastAPI |
| banking-chatbot-rag | RAG chatbot for Turkish banking FAQs with guardrails & compliance. | LangChain · ChromaDB · FastAPI |
📊 Credit risk model — evaluation snapshots
🧰 Side projects & experiments
| Project | Description |
|---|---|
| claude-recipes | Copy-paste-ready skills, subagents, hooks & MCP configs for Claude Code — each one explained (EN/TR). |
| pdf-rag-app | AI-powered PDF Q&A system with RAG & OpenAI. |
| awesome-n8n-workflows | 2 055 production-ready n8n automation templates. |
| price-tracker-bot | Price tracking bot for Trendyol & Amazon with Telegram alerts. |
+ Type-safe Python (Pydantic v2 + mypy --strict on every repo)
+ ≥80% pytest coverage with httpx.MockTransport — no real network calls in CI
+ Conventional commits, ADRs ("Why" + "Trade-offs" + "Revisit if")
+ Async-first I/O (asyncio + httpx + tenacity exponential backoff)
+ Structured JSON logging (structlog) with stable event names
+ Dockerfile + docker-compose + Makefile + GitHub Actions matrix py3.11/3.12
- No untyped public APIs · No secrets in commits · No flaky tests
Every animation on this page is hand-built SVG — no templates. Profile partially maintained by an AI pipeline · co-built with Claude