Code/gguf

Community curated code

github.com

AutoRound is a quantization toolkit for LLMs and VLMs, optimizing performance with high accuracy at low bit widths.

ggufint4llmllmsmxfp4

flux

github.com

Shimmy is a Rust-based inference server providing local, OpenAI-compatible endpoints for machine learning models.

api-servercommand-line-tooldeveloper-toolsggufgpt

flux

github.com

llmfit optimizes large language models for your hardware, ensuring efficient performance and compatibility.

ggufllmlocalaimlxopen-source-coding-agent

flux