Code/cuda

Community curated code

github.com

High-Performance Framework for Language Models

SGLang is an open-source framework for efficient serving of large language and multimodal models, ensuring low-latency and high-throughput performance.

attentionblackwellcudadeepseekdiffusion

flux

github.com

High-Throughput LLM Inference Engine - vLLM

vLLM is an efficient engine for LLM inference and serving, designed for high throughput and memory management.

amdblackwellcudadeepseekdeepseek-v3

flux