Pinned Loading

1. vllm-project/vllm-metal vllm-project/vllm-metal Public

   Community maintained hardware plugin for vLLM on Apple Silicon

   Python 1.5k 202

2. inference-engine-benchmark-system inference-engine-benchmark-system Public

   Production-grade benchmark harness comparing vLLM vs SGLang LLM inference engines across latency, throughput, KV-cache, structured generation, and speculative decoding on NVIDIA A10G (14 models, 2B…

   HTML 1

3. guardrail-compliance-engine guardrail-compliance-engine Public

   Scan Terraform, CloudFormation & Kubernetes for compliance violations. Local checks + AWS Bedrock AI reasoning. SOC 2, CIS, PCI-DSS, HIPAA policies included.

   Python

4. sagemaker-llm-inference-optimizer sagemaker-llm-inference-optimizer Public

   Benchmark and optimize LLM inference backends on Amazon SageMaker for latency, throughput, and cost.

   HTML

5. kv-transfer-tax kv-transfer-tax Public

   Measuring the KV-transfer tax of disaggregated prefill/decode LLM serving (vLLM + NIXL, 4x A10G): on PCIe-only hardware, disagg loses to plain data-parallel replication — measured, committed, repro…

   Python

6. triton-kernel-lab-for-llm-inference triton-kernel-lab-for-llm-inference Public

   Triton GPU kernels for LLM inference: FlashAttention-2 forward, RMSNorm, RoPE, SwiGLU, INT8 matmul, KV-cache ops. Correctness-tested against PyTorch and rooflined on real GPUs (A10G, T4). No invent…

   Python