Open Source LLM Engineering Platform - Langfuse

Langfuse is an open source LLM engineering platform designed to assist teams in the collaborative development, monitoring, evaluation, and debugging of AI applications. It can be self-hosted quickly and is built on the ClickHouse open source database, making it a robust solution for managing large language models (LLMs).

Key features include:

• LLM Application Observability - Track LLM calls and application logic through detailed tracing.
• Prompt Management - Centrally manage and version control prompts with minimal latency.
• Evaluations - Supports various evaluation methods including user feedback and custom pipelines.
• Datasets - Create test sets for continuous improvement and pre-deployment testing.
• LLM Playground - A tool for testing and iterating on prompts and model configurations.

Langfuse also offers comprehensive API support, enabling integration with various frameworks and tools, making it suitable for diverse LLMOps workflows.