Data & MLOps Engineer | Building resilient data pipelines, scalable stream-processing systems, and secured lakehouse platforms where data quality is enforced, not assumed.

Languages

Data Engineering & Streaming

Storage & Query

MLOps & Security

Platform & Serving

A secured medallion lakehouse for European energy trading, with seven security layers and an unsupervised market-abuse surveillance model.

• Tech: Airflow 3 | Spark 4 | PostgreSQL 16 | Vault | scikit-learn | MLflow | DuckDB | Streamlit | Docker
• Architecture Highlights:
  • Medallion lakehouse: Bronze (raw JSON) to Silver (typed, deduplicated) to Gold (PnL, counterparty exposure, portfolio summary), all in Parquet.
  • MLSecOps, layers A to G: gitleaks, Bandit, Semgrep, pip-audit, Trivy, Hadolint, Checkov and OWASP ZAP in CI, plus a data-poisoning gate that runs on live data inside the pipeline.
  • Data as an attack surface: every incoming string is scanned for SQL, script, path-traversal and prompt injection, because the data is later shown to an LLM agent.
  • Quarantine, do not crash: bad rows are set aside rather than failing the run, so a single poisoned value cannot become a denial-of-service.
  • REMIT surveillance model: Isolation Forest, unsupervised because real market abuse is rare and unlabelled. 3/3 planted abuse patterns detected, 5% false-positive rate, and every alert carries a human-readable reason.
  • Found a real bug in production data: the poisoning gate caught an ingestion filter that was fetching grid load in MW instead of price in EUR/MWh. Code review had missed it.

• Focus: Risk-first PDF ingestion, OCR, LLM extraction with Gemini 2.5 Flash, and relational PostgreSQL modeling.
• Architecture Highlights:
  • Risk-First Prototyping: Validated OCR to LLM JSON to Pydantic schema workflow before database design.
  • Complex Entity Relational Mapping: Handled multi-notice PDF complexity by decoupling documents, deeds, companies, and party_roles.
  • Resilience & Idempotency: SHA-256 file hashing for duplicate detection, and exponential backoff retry logic (tenacity) for LLM rate limits.
  • Serving Layer: Clean REST endpoints via FastAPI for business decision-maker lookup and enterprise querying.

• Focus: Change Data Capture pipeline for real-time analytics.
• Tech: Kafka | Flink SQL | Debezium | Redis | OpenSearch | Docker

• Focus: Dynamic masking, HMAC tokenization, and GDPR-compliant sanitized schema generation.
• Tech: PostgreSQL | Python | Docker | Privacy Engineering

• Risk-First Development: Tackle the hardest unknowns, such as data quality and extraction variance, before writing infrastructure code.
• Data Integrity Over Volume: Schema validation, idempotency and explicit error handling come before scaling batch sizes.
• Trust Nothing Upstream: Treat incoming data as an attack surface, not just as input. Validate ranges, scan for injection, and quarantine instead of crashing.
• Root Cause Over Workaround: Verify each fix before moving to the next step. A symptom that disappears is not the same as a cause that is understood.
• Clear System Boundaries: Keep API boundaries clean, avoiding heavy pipeline dependencies in lightweight serving containers.

☕ If a project here (or one of my data-engineering tutorials) saved you time, buying me a coffee helps me keep building and writing.