FastFlowLM: Run LLMs on AMD Ryzen AI NPUs

FastFlowLM (FLM) is a specialized runtime designed to run large language models on AMD Ryzen™ AI NPUs efficiently. It allows users to execute models without the need for a GPU, achieving faster performance while being over 10× more power-efficient. The tool supports context lengths of up to 256k tokens and is lightweight, with an installation size of only 17 MB.

Key features:

• Fast and low power - Operates fully on AMD Ryzen™ AI NPU without burdening GPU or CPU resources.
• Simple CLI and API - Provides a straightforward command-line interface and REST/OpenAI API for ease of use.
• Private and offline - Ensures full privacy as it runs locally without internet access.
• Lightweight runtime - Installs in under 20 seconds, making it easy to integrate into existing workflows.
• No low-level tuning required - Users can focus on application development without needing to adjust model parameters.

FastFlowLM is particularly beneficial for developers looking to leverage local AI capabilities efficiently and effectively.