Top/cuda

The most inspiring discoveries in cuda

github.com

Lucebox: Optimized LLM Inference for RTX 3090

Lucebox is a hub for optimized LLM inference tailored for specific consumer hardware, enhancing AI performance and efficiency.

apple-siliconcppcudakernelllama-cpp

flux

github.com

High-Performance Framework for Language Models

SGLang is an open-source framework for efficient serving of large language and multimodal models, ensuring low-latency and high-throughput performance.

attentionblackwellcudadeepseekdiffusion

flux

github.com

High-Throughput LLM Inference Engine - vLLM

vLLM is an efficient engine for LLM inference and serving, designed for high throughput and memory management.

amdblackwellcudadeepseekdeepseek-v3

flux