Python Framework for Self-Hosted LLM Workflows

Forge is a Python framework designed for self-hosted LLM tool-calling and managing multi-step agentic workflows. It enhances the performance of local models by implementing a reliability layer that includes features like guardrails for error handling and context management.

Key features:

• WorkflowRunner - Allows users to define tools, select a backend, and execute structured agent loops.
• SlotWorker - Provides priority-queued access to a shared inference slot, enabling multiple agents to share a GPU slot efficiently.
• Guardrails middleware - Integrates with user-defined orchestration loops to validate responses and enforce required steps.
• Proxy server - Acts as a drop-in replacement for local model servers, applying guardrails transparently to client interactions.

Forge is particularly useful for developers building applications that require reliable multi-step workflows and context management, making it a valuable tool for AI-driven projects.