Skip to content
View pleyva2004's full-sized avatar
💭
Grinding at 2 am
💭
Grinding at 2 am

Block or report pleyva2004

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
pleyva2004/README.md

Hi there, I'm Pablo Leyva! 👋

Typing SVG

LinkedIn Portfolio Email Profile views


🚀 About Me

I'm a passionate Applied Mathematics & Computer Science student at NJIT with a focus on AI/ML and data science. Currently conducting research on Beyond Next-Token Prediction paradigms for Large Language Models, while leading impactful community organizations.

🎯 Quick Facts

current_focus:
  - Novel LLM architectures (intent-inference over next-token prediction)
  - Advanced transformer architectures & Graph-RAG systems
  - Model Context Protocol (MCP) implementation

collaboration:
  - AI/ML projects
  - Hackathons & competitions
  - Community tech initiatives

achievements:
  - Raised $70K for student organization
  - Led 100+ students to national conferences
  - 2x increase in student internship placements

📫 Let's Connect!

Email LinkedIn Portfolio

💬 Ask Me About

🌱 Currently Learning

  • Advanced MLOps & Model Deployment
  • Graph Neural Networks (GNNs)
  • Quantum Computing Applications
  • Agentic AI Systems

💼 Experience

🍎 Apple - AIML Product Engineering Intern (Summer 2025)

Impact: Led cutting-edge AI/ML product development for Apple Pay

+ Led team of 3 interns building MVP for Agentic Payment flow
+ Prototyped LLM-based product recommendation workflows with Apple Pay integration
+ Implemented Model Context Protocol (MCP) for merchant catalog parsing (TypeScript)
+ Designed Graph-RAG architecture for partner-facing ChatBot

Tech Stack: TypeScript LLM APIs MCP Graph-RAG Vector Databases

🚜 Caterpillar Inc. - Software Engineering Intern (Summer 2024)

Impact: Optimized SDLC using AI-powered analytics and automation

+ Analyzed software development efficiency using Generative AI
+ Built data visualization dashboards for quality metrics tracking
+ Improved team productivity through Agile & DevOps best practices
+ Automated code commit assessment and sprint velocity tracking

Tech Stack: Python Generative AI Agile DevOps Data Visualization

🏆 Featured Projects

Project Description Tech Stack Highlights
🧠 subliminal-research Subliminal Learning: Replication & Mechanism Probes PyTorch torch.func NumPy SciPy Reproduces noise-only trait transfer (0.222 same-init vs 0.122 cross-init); 3 pre-registered hypotheses, 25-seed CIs
📜 scholastic-llm LoRA Fine-Tune for Scholastic Argumentation MLX Qwen 2.5 7B LoRA DPO Claude API 4 adapters published on Hugging Face; best checkpoint 68/120 strict; DPO logged as a clean negative result
⚙️ augustine Local AI Agent Daemon with a Frozen Wire Protocol Python 3.13 Starlette WebSocket SQLite JSON-RPC 2.0 contract never changed across the build; 3 model backends, zero vendor SDKs
📐 first-principles-to-llms Set Theory → Transformers → RLHF, Fully Derived LaTeX PyTorch MLX Jupyter 34 chapters in 3 auto-synchronized forms; definition → theorem → proof → code throughout

🧠 subliminal-research - Does a Trait Survive Distillation on Pure Noise?

What it does: A controlled testbed for subliminal learning — the claim that a student distilled on a teacher's outputs over semantically unrelated data still inherits the teacher's trait. Reproduces the effect on MNIST, then probes the mechanism.

Verified baseline (M = 25 seeds):

  • 🎯 Teacher accuracy 0.944, chance 0.10
  • ✅ Same-init student on pure noise: 0.222
  • ❌ Cross-init student on pure noise: 0.122
  • 📐 Weight-update alignment η_cos: 0.254 vs 0.029

Three pre-registered hypotheses:

  • H1 — tangent-space geometry vs. semantic overlap
  • H2 — smooth continuum vs. hard threshold as inits diverge
  • H3 — bandwidth ceiling vs. sample complexity

The honest headline:

None of the three favored hypotheses earned a clean confirmation — and that is the finding. H1's pre-registered falsifier was actually met, pointing away from the geometry story; the remaining confound is flagged in the writeup rather than buried.

Predictions committed to docs/PREDICTIONS.md before any run, plus a 31-item adversarial code-verification pass.

# Distillation on noise, not MNIST
out = student(bx)[:, :, idx]          # [M,B,L]
with t.no_grad():
    tgt = teacher(bx)[:, :, idx] * teacher_temp

if mode == "soft":
    loss = F.kl_div(
        F.log_softmax(out, -1),
        F.softmax(tgt, -1),
        reduction="batchmean",
    )
# bx is random noise — the student never
# sees a single real digit.

Shared architecture:

mlp:      [784, 256, 256, 13]
logits:   10 real + 3 ghost
seeds:    25 (for CIs)
device:   mps -> cuda -> cpu

Tech Stack: PyTorch NumPy SciPy uv

📄 Ships FINALPAPER.pdf, an executive summary, and a literature review.


📜 scholastic-llm - Teaching Qwen to Argue Like Aquinas

Evaluation (own rubric):

Adapter Strict Balanced
sft-v1
sft-v2-iter400 68/120 68/90
sft-v2
dpo-v3 64/120 63/90

The negative result:

finding: >
  DPO gave no improvement over SFT
  due to preference saturation —
  chosen and rejected samples came
  from the same model family.
status: reported, not hidden

Tech Stack: MLX Qwen HuggingFace Claude

What it does: Fine-tunes Qwen 2.5 7B-Instruct with LoRA (via MLX on Apple Silicon) to answer philosophy and theology questions in a scholastic, Latin-inflected register with citations — a study in register transfer, not a doctrinal authority.

The pipeline:

scrape primary texts  →  clean
   ↓  (Catechism, Summa Theologica,
       Confessions, City of God)
generate pairs w/ Claude as teacher
   ↓
convert to MLX Q8  →  LoRA SFT
   ↓
rubric eval  →  optional DPO

What shipped:

  • 🤗 Four LoRA adapters published on Hugging Face
  • 🎛️ Live Gradio demo on a Hugging Face Space
  • 📄 arXiv-style paper and conference poster, auto-rendered by GitHub Actions on every push
  • 🔬 A post-experiments/ directory of follow-up probes (content-test, layer-frame, math-test), each with its own FINDINGS.md

Status: Complete through Phase 2 (scaled SFT + DPO experiment).


⚙️ augustine - An Agent Harness Built From the Protocol Up

What it does: A from-scratch agent harness. A local daemon runs the full context → model → stream → tools loop and exposes it over a versioned JSON-RPC 2.0 / WebSocket contract; the terminal UI and browser UI are just two clients of that same protocol.

Design decisions worth defending:

  • 📜 The wire contract (schema/wire-v0.json) was frozen up front — and did not change once across the entire build
  • 🔌 Ollama, OpenAI, and Anthropic backends implemented directly on httpxno vendor SDKs
  • 🧱 import-linter enforces ports/adapters in CI: protocol and ports are forbidden from importing any vendor package
  • 🎯 Retrieval-based skill selection scopes each turn to a subset of tools before the model is called (falling back to the full set if scoping would unlock nothing)
  • 💾 Sessions persist to SQLite (WAL) + per-session JSONL transcripts, and replay across a daemon restart

Quality gates: ruff, pyright --strict, import-linter, and pytest all run in CI.

Status: self-labeled v0.1 — shipped. A working prototype with an architecture designed to be extended, not a maintained product.

# The agent turn loop, capped at 8 iters
async def run_agent_turn(
    adapter, registry, messages, emit, *,
    selector=None,
):
    tools = registry.specs() if registry else None
    if selector and tools:
        messages, tools = await _apply_skill_selection(
            selector, messages, tools
        )
    for _ in range(_MAX_TOOL_ITERS):  # = 8
        async for ev in adapter.stream(messages, tools):
            if isinstance(ev, TextChunk):
                await emit(TokenEvent(text=ev.text))
            elif isinstance(ev, ToolCallRequest):
                tool_calls.append(ev)
        if not tool_calls:
            await emit(DoneEvent(reason="complete"))
            return

Tech Stack: Python Starlette Pydantic SQLite


📐 first-principles-to-llms - Nothing Assumed, Everything Derived

% Every chapter, same contract:
\begin{definition} ... \end{definition}
\begin{theorem}    ... \end{theorem}
\begin{proof}      ... \end{proof}
% ...then runnable code.

The 9 blocks:

01. Foundations       # sets, proofs
02. Probability & Info
03. Stochastic Optimization
04. Neural Networks
05. Attention & Transformers
06. Pre-training
07. Post-training     # SFT, RLHF, DPO
08. RL                # MDPs -> GRPO
09. Inference & Serving

Tech Stack: LaTeX PyTorch Jupyter MLX

What it does: A 34-chapter derivation chain that starts at set theory and does not stop until it reaches KV caching and speculative decoding. Every claim is derived, not described.

What makes it unusual:

  • 📚 Every chapter exists in three synchronized forms — Markdown, LaTeX/PDF, and Jupyter — with generate.py verifying all three stay in sync and regenerating TOCs
  • 🤖 CI renders the PDF and HTML automatically on every push
  • 🔨 Chapter 27 ships two GPT pre-training implementations: a pure-NumPy no-dependency baseline (~18K params) and a real PyTorch/MLX GPT (~30M params, 6 layers, d_model=384, tiktoken GPT-2 vocab) on TinyStories
  • 🧮 Recent work has focused on closing logical gaps between chapters — filling in Borel σ-algebra existence, Pinsker's inequality, the performance-difference lemma

Companion repo: math-foundations holds the atomic math reference pages this chain cross-links into.

⚠️ The chapter-27 hardware benchmark table is still a template — the training run is implemented, but the wall-clock and throughput numbers have not been measured yet.

🛠️ Technical Skills

mindmap
  root((Pablo's<br/>Tech Stack))
    Languages
      Python ⭐⭐⭐⭐⭐
      JavaScript/TypeScript ⭐⭐⭐⭐⭐
      Java ⭐⭐⭐⭐
      C++ ⭐⭐⭐⭐
      SQL ⭐⭐⭐⭐
    AI/ML
      PyTorch
      TensorFlow
      Transformers
      LangChain
      RAG Systems
      Vector DBs
    Full Stack
      React/Next.js
      Node.js
      REST APIs
      GraphQL
      WebSockets
    Cloud & DevOps
      AWS
      Google Cloud
      Azure
      Docker
      Kubernetes
      CI/CD
Loading

💻 Programming Languages

Python
Python
JavaScript
JavaScript
TypeScript
TypeScript
Java
Java
C++
C++
R
R
LaTeX
LaTeX

🤖 AI/ML & Data Science

PyTorch
PyTorch
TensorFlow
TensorFlow
scikit-learn
Scikit-learn
Pandas
Pandas
NumPy
NumPy
Matplotlib
Matplotlib
LangChain
LangChain

🌐 Web Development & Frameworks

React
React
Next.js
Next.js
Node.js
Node.js
Express
Express
Flask
Flask
FastAPI
FastAPI
Tailwind
Tailwind

☁️ Cloud & DevOps

AWS
AWS
Google Cloud
GCP
Azure
Azure
Docker
Docker
Kubernetes
Kubernetes
GitHub Actions
GitHub Actions
Git
Git

🗄️ Databases & Tools

PostgreSQL
PostgreSQL
MongoDB
MongoDB
Redis
Redis
Supabase
Supabase
Firebase
Firebase
Prisma
Prisma
GraphQL
GraphQL

📊 GitHub Stats & Activity

Pablo's GitHub Stats Pablo's GitHub Streak
Contribution Graph
📈 More Stats
Top Languages Productive Time
GitHub Trophies

🌟 Leadership & Community

🏛️ President - SHPE NJIT

Society of Hispanic Professional Engineers

team_size: 20+ officers
member_base: 300+ students
funding_raised: $70,000+
conference_attendees: 100+ students

Key Achievements:

  • 🤖 Built CLARA - AI Assistant for operations
    • Email management & auto-response
    • Event planning & logistics
    • Member engagement tracking
  • 📈 2x increase in student internship placements
  • 🎯 Led 100+ students to SHPE National Conference
  • 🤝 Established partnerships with 15+ tech companies

Technologies Used: Python LangChain GPT-4 Gmail API Google Calendar API

🚀 Co-Founder - ALPFA NJIT

Association of Latino Professionals For America

status: First ALPFA chapter at NJIT
focus: Non-engineering Latino professionals
partnerships: 5+ local ALPFA chapters

Key Achievements:

  • 🏢 Established first professional Latino org for non-engineers
  • 🌐 Built external partnerships network
    • Connected with NYC ALPFA chapters
    • Formed corporate partnerships
  • 📚 Organized professional development workshops
  • 🎤 Hosted industry speaker series
  • 💼 Created mentorship program matching students with professionals

Impact: Expanded professional opportunities beyond engineering, creating an inclusive community for all Latino students.

🎯 Leadership Philosophy

"Empowering communities through technology, creating opportunities through collaboration, and building bridges between academia and industry."

Impact Metrics: Students Impacted Funding Raised Events Organized Partnerships

🎯 Current Focus & Research

graph LR
    A[🔬 Research] --> B[Beyond Next-Token Prediction]
    A --> C[Intent-Inference Models]

    D[🤖 AI/ML] --> E[Transformer Architectures]
    D --> F[Graph-RAG Systems]
    D --> G[Agentic AI]

    H[🌐 Community] --> I[SHPE Leadership]
    H --> J[Student Mentorship]
    H --> K[Tech Initiatives]

    L[📚 Learning] --> M[Advanced MLOps]
    L --> N[Graph Neural Networks]
    L --> O[Quantum Computing]

    style A fill:#ff6b6b
    style D fill:#4ecdc4
    style H fill:#ffe66d
    style L fill:#a8e6cf
Loading

🔬 Research Focus

Beyond Next-Token Prediction

Currently exploring novel paradigms for Large Language Models that reframe training from simple next-token prediction toward intent-inference and reasoning capabilities.

Key Areas:

  • 🧠 Intent-based learning architectures
  • 🔄 Interactive reasoning systems
  • 📊 Multi-modal transformers
  • 🎯 Context-aware embeddings

🚀 Active Projects

Building the Future of AI

  • 🤖 CLARA 2.0: Enhanced AI assistant with advanced reasoning
  • 📊 Graph-RAG Framework: Novel retrieval architecture for LLMs
  • 🔗 MCP Implementations: Model Context Protocol integrations
  • 🎓 ML Education Platform: Teaching AI/ML to fellow students

💬 Let's Connect!


📫 Reach Out


Professional Networking

View My Work

Send a Message

📊 Profile Analytics

Profile Views GitHub Followers GitHub Stars


💡 Open to Opportunities

interests:
  - AI/ML Research Collaborations
  - Open Source Contributions
  - Hackathons & Competitions
  - Speaking Engagements
  - Mentorship Opportunities

availability:
  status: "Open to interesting projects!"
  best_for: "AI/ML, Full-Stack, Research"
  response_time: "Usually within 24 hours"



⭐️ From pleyva2004 | Built with ❤️ and lots of ☕


Popular repositories Loading

  1. Clara Clara Public

    This repository was made with the intentions of assisting on Campus Organizations with email management. Allowing students to focus on academics and influence.

    Python 1

  2. claude-skill-study-paper claude-skill-study-paper Public

    Claude Code skill that turns an AI/ML paper into 5 layered artifacts: interview prep, math deep dive, opinion capture, GitHub sandbox, and a research-ready LaTeX lit-review entry.

    Python 1

  3. rethinking-rl-llm-reasoning-sparse-policy-selection rethinking-rl-llm-reasoning-sparse-policy-selection Public

    Sandbox study of: Rethinking RL for LLM Reasoning (Akgül et al. 2026, arxiv:2605.06241).

    TeX 1

  4. elf-embedded-language-flows elf-embedded-language-flows Public

    Layered study of ELF: Embedded Language Flows (Hu et al. 2026, arxiv:2605.10938). Continuous flow matching in a pretrained embedding space with weight-tied unembedding for final-step discretization.

    TeX 1

  5. first-principles-to-llms first-principles-to-llms Public

    Unbroken chain of derivation: sets → analysis → linear algebra → probability → optimization → neural networks → attention → transformer → pre-training → SFT → RLHF/DPO

    HTML 1

  6. scholastic-llm scholastic-llm Public

    Fine-tuning Qwen 2.5 to debate philosophy in a scholastic register, grounded in the Catechism of the Catholic Church (CCC)

    Python 1