Skip to content
View LeonelSalvo's full-sized avatar
🎯
Focusing
🎯
Focusing

Block or report LeonelSalvo

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
LeonelSalvo/README.md
Leonel Salvo — Lead AI Developer · neural networks from first principles

pinguinista.com LinkedIn Email profile views


🧠 About

Lead AI Developer — production AI infrastructure by day, neural networks rebuilt from scratch by night, so I actually understand what I ship.

Full-Stack Engineer  ➜  Product Discovery Lead  ➜  Lead AI Developer


Across AI engineering, games and tools, one question: can I explain and intervene in what I build?

  • 🔬  Research — I rebuild modern architectures from first principles: no nn.Transformer, no wrapper I can't explain.
  • 🤖  Work — I lead a private, multi-tenant AI platform: composable agents + RAG over a customer's own documents, in production.
  • 🎮  Games — a one-person Godot studio, Pinguinista: Una última cosa in development; Records of the Truth Bureau on hold. Neither released.
  • 🧰  ToolsInvolut, applying the Auctor method to learn from work produced with coding agents.
  • 📚  Permanent student: a degree in hardware / electronics, and pushing from LLMs toward CUDA and quantum.

🔬 Research — LLMs from scratch

The modern stack rebuilt one self-checking component at a time, in PyTorch, on a single RTX 3090.


modern-nanoGPT
dense transformer · RMSNorm · RoPE · SwiGLU · GQA

stars

The modern dense transformer. The GPT-2 skeleton upgraded to the 2026 open-weight stack — RMSNorm · RoPE · SwiGLU · GQA · tied weights — built one self-checking component at a time. Trained on a single RTX 3090 (val ≈ 1.48).

nano-moe-mla
sparse MoE + MLA · routing probe · ablation

stars

The sparse template: MoE + MLA in one model. Built from scratch with the instruments to study it — a labeled multi-domain corpus, a router-specialization probe (mutual information), and a seed-averaged stack ablation that confirms the balancing ↔ specialization tradeoff.

frontier-llm-techniques-2026-Q1
Muon · Multi-Token Prediction · BPE

stars

Frontier techniques, from scratch. The Muon optimizer (used to train Kimi K2) and Multi-Token Prediction (DeepSeek-V3, Gemma 4, GLM, Qwen), plus a base BPE tokenizer — each self-contained and self-checking, with an AdamW-vs-Muon benchmark.

🤖 Work

I lead a private, multi-tenant enterprise AI platform — composable agents and retrieval over a customer's own documents, shipped to production. (Architecture kept private.)


🎮 Games — Pinguinista


🐧 Pinguinista
my one-person game studio

Two projects, neither released. Una última cosa is in active development; Records of the Truth Bureau is on hold. Gameplay and screenshots are on pinguinista.com.

Records of the Truth Bureau
noir detective · writing to think

On hold. A noir detective game about evidence, writing and reasoning.

Una última cosa
Farming survival horror

In development. Care for your home and community by day. Pray for God’s mercy at night. Current gameplay and screenshots are available on the game page.

🧰 Understand what you build. Grow through the work.

Augmented Mastery gives the purpose. Auctor proposes the practice. Involut applies it to code.

Augmented Mastery is the philosophy: delegating execution can coexist with developing understanding, judgment and the ability to intervene in your work.

Auctor is the method under development: get oriented, take a position, check the evidence and revisit what you learned. Agreement with an agent is not proof of mastery. The method is a draft; its learning outcomes still need evaluation.

Involut is the product applying it to code: decision cards, questions before revealing, access to original material and notes for the next exchange with your agent. TypeScript · React · three.js. The product repository remains private during development; no public demo is claimed.

Explore the philosophy, method and product · Read the manifesto · Document repository

Manifesto and Auctor texts: CC BY-SA 4.0. Involut software: AGPLv3. Separate projects, with separate licensing scopes.


🧬 Reproduced from scratch

I didn't invent these — I learned them by rebuilding and verifying each one in code, following Karpathy's Zero to Hero and the open-model papers.

Built What it is Learned from
micrograd a tiny autograd engine — backprop by hand Karpathy · Zero to Hero
makemore MLP, BatchNorm, manual backprop, WaveNet Karpathy · Zero to Hero
self-attention the attention mechanism, from the math up Attention Is All You Need
modern-nanoGPT the modern dense transformer (RMSNorm · RoPE · GQA · SwiGLU) nanoGPT + Llama / Mistral / Qwen
nano-moe-mla sparse MoE + MLA in one model, with a routing probe + ablation DeepSeek-V2/V3 papers
Muon optimizer orthogonalized-momentum (Newton-Schulz) + a benchmark Keller Jordan · Moonshot (Kimi K2)
Multi-Token Prediction a 2nd head predicting t+2 — a denser training signal DeepSeek-V3
BPE tokenizer byte-pair encoding, exact round-trip Karpathy · minBPE

Method: from-scratch first; a wrapper is only allowed once I can name what it wraps and the trade-off.


📖 Learning path

✅ Done

  • Karpathy — Neural Networks: Zero to Hero (10/10)
  • Stanford CS229 — Machine Learning (Andrew Ng)
  • fast.ai — Practical Deep Learning, Part 1

🔄 In progress

  • Hugging Face — LLM Course (transformers, fine-tuning, reasoning)
  • Reinforcing attention internals (RoPE · KV-cache · RMSNorm)

⏭️ Next

  • nanochat (Karpathy) — the full ChatGPT pipeline
  • Raschka — Build a Reasoning Model (From Scratch)

🧪 Later

  • Triton → GPU-MODE + PMPP (CUDA kernels)
  • Mamba / SSM hybrids · Diffusion LLMs · Quantum

🛠️ Stack

AI / ML  

Backend  

Frontend  

Infra & also  


📊 Stats

followers stars profile views



pinguinista.com

Pinned Loading

  1. modern-nanoGPT modern-nanoGPT Public

    From-scratch PyTorch GPT: the GPT-2 skeleton plus the components of current open LLMs — RMSNorm, RoPE, SwiGLU, GQA, no-bias + tied embeddings. Each isolated and self-checking.

    Python

  2. frontier-llm-techniques-2026-Q1 frontier-llm-techniques-2026-Q1 Public

    From-scratch PyTorch: frontier LLM techniques as of 2026-Q1 — the Muon optimizer and Multi-Token Prediction, plus a base BPE tokenizer. Self-contained, self-checking modules.

    Python 2 1

  3. nano-moe-mla nano-moe-mla Public

    DeepSeek-style MoE + MLA from scratch in PyTorch, with a router-specialization probe (mutual information) and a dense/+MoE/+MLA ablation. Educational, nano-scale, measured.

    Python 2 1