FlagOS is a unified, open-source AI system software stack designed for multi-chip scenarios. It was jointly initiated and established by over ten domestic and international organizations, including chip companies, system manufacturers, algorithm and software-related entities, non-profit organizations, and research institutions.
Addressing core pain points in utilizing diverse AI chips, FlagOS builds a comprehensive system software ecosystem that demonstrates the potential to break down ecosystem barriers between different chip software stacks, effectively reducing migration costs for developers.
Components are grouped by the SIG that owns them. See the SIG overview for charters, owners, and meeting calendars.
| SIG | Purpose | Repository |
|---|---|---|
| sig-operator | Operator libraries: implementation, performance optimization, and multi-chip adaptation, including framework-facing fused operator packages |
FlagGems FlagGems-vllm FlagGems-sglang FlagAttention FlagFFT FlagSparse FlagDNN FlagBLAS FlagTensor FlagAudio |
| sig-compiler | Unified AI compiler: IR design, optimization passes, and multi-chip code generation | FlagTree |
| sig-network | Unified communication library: collective communication and multi-chip, multi-node tuning | FlagCX |
| sig-framework | Framework adapter layer for PyTorch, vLLM, SGLang, TransformerEngine, Megatron-LM, and veRL |
Torch-FL vllm-plugin-FL sglang-plugin-FL TransformerEngine-FL Megatron-LM-FL verl-FL |
| sig-training | Training and inference orchestration: parallelism strategies, recipes, and Hugging Face integration | FlagScale |
| sig-kernelgen | AI-assisted kernel code generation and its evaluation |
KernelGen KernelGenBench |
| sig-chip | Datacenter chip adaptation: vendor SDK integration, bring-up, and multi-chip CI | No dedicated repository; adaptation lands in the module repositories above |
| wg-embodied (incubating) | Robotics and embodied intelligence | FlagOS-Robo |
| wg-ai4s (incubating) | AI for Science, including quantum simulation | FlagQuantum |
| sig-tools (planned) | Release management toolchain | FlagRelease |
| sig-agent (planned) | Skill packages for agents to complete tasks in a specific domain | skills |
| sig-benchmark (planned) | Multi-chip performance benchmarking and evaluation | FlagPerf |
-
KernelGenBench: A benchmark framework for evaluating LLM and agent-based Triton kernel generation across multiple hardware platforms.
-
libtriton_jit: Triton JIT C++ runtime for reduced Python overhead.
-
Megatron-LM-FL: GPU-optimized library for training transformer models at scale
-
TransformerEngine-FL: FP8 mixed precision training for transformer models
-
verl-FL: A fork of verl designed to support diverse AI accelerators, built on top of FlagOS.
-
Torch-FL: A custom PyTorch device plugin based on the
PrivateUse1extension mechanism, registering FlagGems high-performance Triton operators as the flagos device backend for unified multi-chip support.
-
vLLM-plugin-FL: Advanced vLLM plugin for enhanced multi-chip support
-
sglang-plugin-FL: An out-of-tree (OOT) plugin for SGLang, built on FlagOS's unified multi-chip backend
-
FlagDNN: A deep neural network computing library oriented towards multiple chip backends.
-
FlagBLAS: A computing library that follows the BLAS standard interface and is oriented towards multiple chip backends.
-
FlagFFT: Efficient algorithms that convert signals or data between the time/spatial domain and the frequency domain in operations targeting multiple chip backends.
-
FlagSparse: A package for sparse operations (SpMV, gather, scatter, sparse formats) targeting multiple chip backends.
-
FlagTensor: A package for high-performance tensor operators targeting multiple chip backends.
-
FlagAudio: A package for high-performance audio data processing targeting multiple chip backends.
-
FlagAttention: A package for memory-efficient attention operators, initiated as an extension of multi-head attention.
-
FlagGems-vLLM: A collection of optimized implementations of common vLLM operators and supports high-performance inference and deployment for a variety of widely used models.
-
EasyOfUse: Simplify development for open compute - plug-and-play solutions and best practices.
-
Community: FlagOS community governance, contribution guides, and communication hub.
- FlagOS Wiki & Documentation: DeepWiki Portal
- Latest Release v1.5: Announcement
- ModelScope: FlagRelease Models
- Hugging Face: FlagRelease Models
- WiseModel: FlagRelease Models
- WeChat Official Account: FlagOpen
- WeChat Channels Account: FlagOpen
Ready to contribute or learn more? Visit the FlagOS Community Repository for:
- 📖 Comprehensive contribution guidelines
- 💬 Community discussions and support
- 🤝 Ways to get involved
- 📚 Resources and documentation
We welcome contributions from the community! Whether through starring, forking, or submitting pull requests, your involvement helps make FlagOS better.
- ⭐ Star - Show your support!
- 🔔 Watch - Stay updated with our latest releases
- 🍴 Fork - Start contributing!
Please read and respect our Code of Conduct:
- Code of Conduct (English)
FlagOS and its components are licensed under the Apache License 2.0. See individual repositories for details.
FlagOS is jointly supported by leading organizations in chip manufacturing, system software, and AI research. This collaborative effort demonstrates the commitment to building an open, unified ecosystem for AI system software across diverse hardware platforms.
Join us in building the future of AI system software! 🚀