DeepEval vs Langfuse

The LLM Evaluation Framework

Documentation | Metrics and Features | Getting Started | Integrations | Confident AI

<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=de">Deutsch</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=es">Español</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=fr">français</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=ja">日本語</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=ko">한국어</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=pt">Português</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=ru">Русский</a> |
<a href="https://www.readme-i18n.com/confident-ai/deepeval?lang=zh">中文</a>

DeepEval is a simple-to-use, open-source LLM evaluation framework, for evaluating large-language model systems. It is similar to Pytest but specialized for unit testing LLM apps. DeepEval incorporates the latest research to run evals via metrics such as G-Eval, task completion, answer relevancy, hallucination, etc., which use LLM-as-a-judge and other NLP models that run locally on your machine.

Whether you're building AI agents, voice agents, multi-turn chatbots, RAG pipelines, or anything else, implemented via LangChain or OpenAI, DeepEval has you covered. With it, you can easily evaluate:

• LLM apps end-to-end as black boxes
• Complete AI agent trajectories across every decision and action
• Individual AI agent steps such as LLM calls, tool use, retrieval, and sub-agent handoffs

Use these evaluations to determine the optimal models, prompts, and architecture to improve your AI quality, prevent prompt drifting, or even transition from OpenAI to Claude with confidence.

Jev is now available in DeepEval, both in existing metrics and the latest JevEval.

[!IMPORTANT]
If you're already in production, Confident AI surfaces failing traces before anyone has written an eval for them, clusters them into failure modes, and turns them into your next eval.

🔥 Metrics and Features

• 📐 Large variety of ready-to-use LLM eval metrics (all with explanations) powered by ANY LLM of your choice, statistical methods, or NLP models that run locally on your machine covering all use cases:

  • Custom, All-Purpose Metrics:

    • JevEval — a custom metric scored by Jev, a System One model that answers your questions with calibrated probabilities instead of generated text
    • G-Eval — a research-backed LLM-as-a-judge metric for evaluating on any custom criteria with human-like accuracy
    • DAG — DeepEval's graph-based deterministic LLM-as-a-judge metric builder
  • AI Agent Metrics

    • Task Completion — evaluate whether an AI agent accomplished its goal
    • Tool Correctness — check if the right tools were called with the right arguments
    • Goal Accuracy — measure how accurately the agent achieved the intended goal
    • Step Efficiency — evaluate whether the agent took unnecessary steps
    • Plan Adherence — check if the agent followed the expected plan
    • Plan Quality — evaluate the quality of the agent's plan
    • Tool Use — measure quality of tool usage
    • Argument Correctness — validate tool call arguments
  • Voice Agent Metrics

    • Agent Responsiveness — check whether the voice agent answered when it was spoken to
    • Turn-Taking Naturalness — measure response latency and interruptions across the call
    • Voice Naturalness — evaluate how natural the voice agent sounds
    • Transcription Accuracy — check whether the voice agent's speech-to-text heard the caller correctly
    • Audio Glitch — detect audio artifacts in the voice agent's speech
  • Multi-Turn Metrics

    • Knowledge Retention — evaluate whether the chatbot retains factual information throughout a conversation
    • Conversation Completeness — measure whether the chatbot satisfies user needs throughout a conversation
    • Turn Relevancy — evaluate whether the chatbot generates consistently relevant responses throughout a conversation
    • Turn Faithfulness — check if the chatbot's responses are factually grounded in retrieval context across turns
    • Role Adherence — evaluate whether the chatbot adheres to its assigned role throughout a conversation
  • RAG Metrics

    • Answer Relevancy — measure how relevant the output is to the input
    • Faithfulness — evaluate whether the RAG pipeline's output factually aligns with the retrieval context
    • Contextual Recall — measure how well the RAG pipeline's retrieval context aligns with the expected output
    • Contextual Precision — evaluate whether relevant nodes in the RAG pipeline's retrieval context are ranked higher
    • Contextual Relevancy — measure the overall relevance of the RAG pipeline's retrieval context to the input
    • RAGAS — average of answer relevancy, faithfulness, contextual precision, and contextual recall
  • MCP Metrics

    • MCP Task Completion — evaluate how effectively an MCP-based agent accomplishes a task
    • MCP Use — measure how effectively an agent uses its available MCP servers
    • Multi-Turn MCP Use — evaluate MCP server usage across conversation turns
  • Multimodal Metrics

    • Text to Image — evaluate image generation quality based on semantic consistency and perceptual quality
    • Image Editing — evaluate image editing quality based on semantic consistency and perceptual quality
    • Image Coherence — measure how well images align with their accompanying text
    • Image Helpfulness — evaluate how effectively images contribute to user comprehension of the text
    • Image Reference — evaluate how accurately images are referred to or explained by accompanying text
  • Other Metrics

    • Hallucination — check whether the LLM generates factually correct information against provided context
    • Summarization — evaluate whether summaries are factually correct and include necessary details
    • Bias — detect gender, racial, or political bias in LLM outputs
    • Toxicity — evaluate toxicity in LLM outputs
    • JSON Correctness — check whether the output matches an expected JSON schema
    • Prompt Alignment — measure whether the output aligns with instructions in the prompt template

• 🎯 Supports both end-to-end and component-level LLM evaluation.

• 🧩 Build your own custom metrics that are automatically integrated with DeepEval's ecosystem.

• 🔮 Generate single and multi-turn synthetic datasets, and simulate live voice calls against your voice agent.

• 🔗 Integrates seamlessly with ANY CI/CD environment.

• 🧬 Optimize prompts automatically based on evaluation results.

• 🏆 Easily benchmark ANY LLM on popular LLM benchmarks in under 10 lines of code., including MMLU, HellaSwag, DROP, BIG-Bench Hard, TruthfulQA, HumanEval, GSM8K.

🔌 Integrations

DeepEval plugs into any LLM framework — OpenAI Agents, LangChain, CrewAI, and more — and into voice stacks like Vapi, ElevenLabs, LiveKit, and Pipecat. The same traces can go to Confident AI once you're live, where evals run on every production trace.

Frameworks

• LangChain — evaluate LangChain applications with a callback handler
• LangGraph — evaluate LangGraph agents with a callback handler
• Pydantic AI — evaluate Pydantic AI agents with type-safe validation
• CrewAI — evaluate CrewAI multi-agent systems
• Anthropic — evaluate and trace Claude applications via a client wrapper
• AWS AgentCore — evaluate agents deployed on Amazon AgentCore
• Google ADK — evaluate Google ADK agents and multi-agent workflows
• AI SDK — evaluate AI SDK generations and tool-loop trajectories
• Mastra — evaluate Mastra agents and workflows with native tracing
• OpenAI — evaluate and trace OpenAI applications via a client wrapper
• OpenAI Agents — evaluate OpenAI Agents end-to-end in under a minute
• LlamaIndex — evaluate AI agents and RAG applications built with LlamaIndex

Voice Agent Connectors

• Vapi — simulate calls to a Vapi assistant over its WebSocket transport
• ElevenLabs — simulate calls to an ElevenLabs conversational agent
• LiveKit — join your agent's LiveKit room and trade audio over WebRTC
• Pipecat — connect to your Pipecat pipeline over its WebSocket transport

☁️ Platform + Ecosystem

Confident AI is AI monitoring for applications in production. DeepEval tests what you know to test before you ship; Confident AI finds what you didn't, and it's model- and framework-agnostic.

• Failing traces surface before anyone has written an eval for them, clustered into failure modes, with user sentiment and behavior across traces.
• Failing traces route to the right annotators and become goldens, so production issues turn into your next eval.
• For enterprise teams, one system for AI quality: evals, monitoring, human annotation, and voice-native observability, with governance as release gates.
• Prefer to stay in your editor? Inspect failing traces and pull the datasets built from them in Claude Code or Cursor through Confident AI's MCP server.

🤖 Vibe-Coder QuickStart

Want your coding agent to add evals and fix failures for you? Install the DeepEval skill, point it at your AI agent, voice agent, multi-turn chatbot, RAG pipeline, or any other LLM app, and ask it to generate a dataset, write the eval suite, run deepeval test run, and iterate on the failing metrics.

Start with the 5-minute vibe-coder guide.

🚀 Human QuickStart

Let's pretend your LLM application is a customer support AI agent; here's how DeepEval can help test what you've built.

Installation

Deepeval works with Python>=3.10+.

pip install -U deepeval

Create an account (highly recommended)

Using the deepeval platform will allow you to generate sharable testing reports on the cloud. It is free, takes no additional code to setup, and we highly recommend giving it a try.

To login, run:

deepeval login

Follow the instructions in the CLI to create an account, copy your API key, and paste it into the CLI. All test cases will automatically be logged (find more information on data privacy here).

Write your first test case

Create a test file:

touch test_ai_agent.py

Open test_ai_agent.py and write your first test case to run an end-to-end evaluation using DeepEval, which treats your LLM app as a black-box:

import pytest
from deepeval import assert_test
from deepeval.metrics import GEval
from deepeval.test_case import LLMTestCase, SingleTurnParams

def test_case():
    correctness_metric = GEval(
        name="Correctness",
        criteria="Determine if the 'actual output' is correct based on the 'expected output'.",
        evaluation_params=[SingleTurnParams.ACTUAL_OUTPUT, SingleTurnParams.EXPECTED_OUTPUT],
        threshold=0.5
    )
    test_case = LLMTestCase(
        input="What if these shoes don't fit?",
        # Replace this with the actual output from your LLM application
        actual_output="You have 30 days to get a full refund at no extra cost.",
        expected_output="We offer a 30-day full refund at no extra costs.",
    )
    assert_test(test_case, [correctness_metric])

Set your OPENAI_API_KEY as an environment variable (you can also evaluate using your own custom model, for more details visit this part of our docs):

export OPENAI_API_KEY="..."

And finally, run test_ai_agent.py in the CLI:

deepeval test run test_ai_agent.py

Congratulations! Your test case should have passed ✅ Let's break down what happened.

• The variable input mimics a user input, and actual_output is a placeholder for what your application's supposed to output based on this input.
• The variable expected_output represents the ideal answer for a given input, and GEval is a research-backed metric provided by deepeval for you to evaluate your LLM outputs on any custom criteria with human-like accuracy.
• In this example, the metric criteria is correctness of the actual_output based on the provided expected_output.
• All metric scores range from 0 - 1, which the threshold=0.5 ultimately determines if your test has passed or not.

Read our documentation for more information!

Evals With Full Traceability

Use evals_iterator() to run the same dataset through your app, whether you instrument it manually or through one of DeepEval's framework integrations. Because tracing captures the ordered sequence of model decisions, tool calls, and intermediate steps, you can run trajectory-based evaluations over the complete path your AI agent takes.

Here's an example of manual instrumentation:

from deepeval.tracing import observe, update_current_span
from deepeval.test_case import LLMTestCase
from deepeval.metrics import TaskCompletionMetric

@observe()
def inner_component(input: str):
    output = "result"
    update_current_span(test_case=LLMTestCase(input=input, actual_output=output))
    return output

@observe()
def app(input: str):
    return inner_component(input)

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    app(golden.input)

from deepeval.openai import OpenAI
from deepeval.tracing import trace
from deepeval.metrics import TaskCompletionMetric

client = OpenAI()

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator():
    with trace(metrics=[TaskCompletionMetric()]):
        client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": golden.input}],
        )

from agents import Runner
from deepeval.metrics import TaskCompletionMetric

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    Runner.run_sync(agent, golden.input)

import { generateText } from "ai";
import { configureAiSdkTracing } from "deepeval/integrations/ai-sdk";
import { TaskCompletionMetric } from "deepeval/metrics";

const tracer = configureAiSdkTracing({ name: "my-agent" });

const ask = (input: string) =>
  generateText({
    model,
    prompt: input,
    experimental_telemetry: { isEnabled: true, tracer },
  });

// This metric evaluates the complete trajectory captured for this run.
for await (const golden of dataset.evalsIterator({
  metrics: [new TaskCompletionMetric()],
})) {
  await ask(golden.input);
}

import { Mastra } from "@mastra/core/mastra";
import { Observability } from "@mastra/observability";
import { DeepEvalExporter } from "deepeval/integrations/mastra";
import { TaskCompletionMetric } from "deepeval/metrics";

const mastra = new Mastra({
  agents: { agent },
  observability: new Observability({
    configs: {
      deepeval: { exporters: [new DeepEvalExporter()] },
    },
  }),
});

// This metric evaluates the complete trajectory captured for this run.
for await (const golden of dataset.evalsIterator({
  metrics: [new TaskCompletionMetric()],
})) {
  await mastra.getAgent("agent").generate(golden.input);
}

from deepeval.anthropic import Anthropic
from deepeval.tracing import trace
from deepeval.metrics import TaskCompletionMetric

client = Anthropic()

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator():
    with trace(metrics=[TaskCompletionMetric()]):
        client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=1024,
            messages=[{"role": "user", "content": golden.input}],
        )

from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator():
    llm.invoke(
        golden.input,
        config={"callbacks": [CallbackHandler(metrics=[TaskCompletionMetric()])]},
    )

from deepeval.integrations.langchain import CallbackHandler
from deepeval.metrics import TaskCompletionMetric

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator():
    agent.invoke(
        {"messages": [{"role": "user", "content": golden.input}]},
        config={"callbacks": [CallbackHandler(metrics=[TaskCompletionMetric()])]},
    )

from deepeval.metrics import TaskCompletionMetric

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    agent.run_sync(golden.input)

from deepeval.integrations.crewai import instrument_crewai
from deepeval.metrics import TaskCompletionMetric

instrument_crewai()

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    crew.kickoff({"input": golden.input})

from deepeval.integrations.agentcore import instrument_agentcore
from deepeval.metrics import TaskCompletionMetric

instrument_agentcore()

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    invoke({"prompt": golden.input})

import asyncio
from deepeval.evaluate.configs import AsyncConfig
from deepeval.metrics import TaskCompletionMetric

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(
    async_config=AsyncConfig(run_async=True),
    metrics=[TaskCompletionMetric()],
):
    task = asyncio.create_task(agent.run(golden.input))
    dataset.evaluate(task)

import asyncio
from deepeval.evaluate.configs import AsyncConfig
from deepeval.integrations.google_adk import instrument_google_adk
from deepeval.metrics import TaskCompletionMetric

instrument_google_adk()

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(
    async_config=AsyncConfig(run_async=True),
    metrics=[TaskCompletionMetric()],
):
    task = asyncio.create_task(run_agent(golden.input))
    dataset.evaluate(task)

from deepeval.integrations.strands import instrument_strands
from deepeval.metrics import TaskCompletionMetric

instrument_strands()

## This metric evaluates the complete trajectory captured for this run.
for golden in dataset.evals_iterator(metrics=[TaskCompletionMetric()]):
    agent(golden.input)

Learn more about component-level evaluations here.

Evaluate Without Pytest Integration

Alternatively, you can evaluate without Pytest, which is more suited for a notebook environment.

from deepeval import evaluate
from deepeval.metrics import AnswerRelevancyMetric
from deepeval.test_case import LLMTestCase

answer_relevancy_metric = AnswerRelevancyMetric(threshold=0.7)
test_case = LLMTestCase(
    input="What if these shoes don't fit?",
    # Replace this with the actual output from your LLM application
    actual_output="We offer a 30-day full refund at no extra costs.",
    retrieval_context=["All customers are eligible for a 30 day full refund at no extra costs."]
)
evaluate([test_case], [answer_relevancy_metric])

Using Standalone Metrics

DeepEval is extremely modular, making it easy for anyone to use any of our metrics. Continuing from the previous example:

from deepeval.metrics import AnswerRelevancyMetric
from deepeval.test_case import LLMTestCase

answer_relevancy_metric = AnswerRelevancyMetric(threshold=0.7)
test_case = LLMTestCase(
    input="What if these shoes don't fit?",
    # Replace this with the actual output from your LLM application
    actual_output="We offer a 30-day full refund at no extra costs.",
    retrieval_context=["All customers are eligible for a 30 day full refund at no extra costs."]
)

answer_relevancy_metric.measure(test_case)
print(answer_relevancy_metric.score)
## All metrics also offer an explanation
print(answer_relevancy_metric.reason)

Note that some metrics are built for specific use cases, such as AI agents, voice agents, multi-turn chatbots, RAG, and more. Make sure to use our docs to pick the right one for your use case.

A Note on Env Variables (.env / .env.local)

DeepEval auto-loads .env.local then .env from the current working directory at import time.
Precedence: process env -> .env.local -> .env.
Opt out with DEEPEVAL_DISABLE_DOTENV=1.

cp .env.example .env.local
## then edit .env.local (ignored by git)

DeepEval With Confident AI

Confident AI monitors AI applications in production. Evals run on every production trace, failing traces are clustered into failure modes, and each one can be routed to the right annotators — engineers, QA, domain experts or PMs — and turned into a golden for your next eval. The traces your app already produces with DeepEval stream to it with no code changes.

Log in from the CLI to connect:

deepeval login

Test runs also appear on the platform once you're logged in.

Prefer to stay in your IDE? Inspect failing traces and pull the datasets built from them through Confident AI's MCP server without leaving your editor.

Learn more in the Confident AI documentation.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Roadmap

Features:

• Integration with Confident AI
• Implement G-Eval
• Implement RAG metrics
• Implement Conversational metrics
• Evaluation Dataset Creation
• Red-Teaming
• DAG custom metrics
• Guardrails

Authors

Built by the founders of Confident AI. Contact jeffreyip@confident-ai.com for all enquiries.

License

DeepEval is licensed under Apache 2.0 - see the LICENSE.md file for details.

alt="chat on Discord"> alt="follow on X(Twitter)">

🧑‍💻 We're hiring

Interested in contributing to Langfuse full-time? Langfuse is growing fast (we doubled the team in the last 6 months). Since January 2026 we're part of ClickHouse. We hire remotely, with offices in Berlin, San Francisco, Amsterdam, and New York.
See open roles →

✨ Core Features

• AI Observability: Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions. Built on OpenTelemetry. Try the interactive demo to see this in action.

• Prompt Management helps you centrally manage, version control, and collaboratively iterate on your prompts. Thanks to strong caching on server and client side, you can iterate on prompts without adding latency to your application. The LLM Playground lets you test prompts and model configurations; when you see a bad result in tracing, you can jump straight to the playground to iterate on it.

• Skill Management (beta) brings versioning and labels to agent skills: a SKILL.md with task instructions plus optional supporting files. Promote or roll back versions without changing application code, and link skill versions to the traces they produced.

• Evaluations are key to the AI agent development workflow, and Langfuse adapts to your needs. It supports LLM-as-a-judge, code evaluators, user feedback collection, manual labeling, and custom evaluation pipelines via APIs/SDKs. Use Datasets and Experiments to build test sets and benchmarks for pre-deployment testing and continuous improvement.

• AI Gateway (beta) gives your organization one endpoint for calls to your model providers. Keep provider credentials in one place, hand out per-user or service account gateway keys, approve models centrally, track AI spend per user, key, and model, and record every call as a Langfuse trace.

• Metrics and Dashboards: Track latency, cost, and quality across any dimension with custom dashboards, the metrics API, and alerts.

• Comprehensive API: Langfuse is frequently used to power bespoke AI engineering workflows while using the building blocks provided by Langfuse via the API. An OpenAPI spec, typed SDKs for Python and JS/TS, the Langfuse CLI, and the Langfuse Agent Skill for coding agents are available.

📦 Deploy Langfuse

Langfuse Cloud

Managed deployment by the Langfuse team, generous free tier, no credit card required.

Sign up for Langfuse Cloud →

Self-Host Langfuse

Run Langfuse on your own infrastructure:

• Docker Compose: Run Langfuse on your own machine or a single virtual machine in 5 minutes. The guide also covers configuring Docker log rotation so container logs don't fill up the disk.

  # Get a copy of the latest Langfuse repository
  git clone --depth=1 https://github.com/langfuse/langfuse.git
  cd langfuse
  
  # Run the langfuse docker compose
  docker compose up
  

  Copy

• Kubernetes (Helm): Run Langfuse on a Kubernetes cluster using Helm. This is the preferred production deployment.

• Terraform Templates: AWS, Azure, GCP

• Community-maintained: Render, Railway

See self-hosting documentation to learn more about architecture and configuration options.

[!TIP]
Self-hosting Langfuse? Subscribe to the self-hosting update list to get an email about important features and new releases for open source Langfuse — self-hosting updates only, no marketing.

🔌 Integrations

Langfuse is built on OpenTelemetry, so any OpenTelemetry-instrumented application, framework, or collector can send traces to Langfuse.

Main Integrations

Packages integrated with Langfuse

See the integrations overview for the full list of 100+ integrations.

🚀 Quickstart

Instrument your app and start ingesting traces to Langfuse, thereby tracking LLM calls and other relevant logic in your app such as retrieval, embedding, or agent actions. Inspect and debug complex logs and user sessions.

🤖 Agentic installation (recommended)

Install the Langfuse Agent Skill to let your coding agent access all Langfuse features, then ask it to add tracing to your application:

Install the Langfuse Agent Skill from github.com/langfuse/skills
and use it to add tracing to this application with Langfuse
following best practices.

You can also install the skill with the skills CLI (npx skills add langfuse/skills --skill "langfuse") or the Langfuse Cursor plugin. If your agent supports MCP, add the Langfuse Docs MCP server (https://langfuse.com/api/mcp, no authentication) so it can look up current documentation while it works. See the get started guide for details.

🛠️ Manual installation

1️⃣ Create new project

1. Create Langfuse account or self-host
2. Create a new project
3. Create new API credentials in the project settings

2️⃣ Log your first LLM call

The @observe() decorator makes it easy to trace any Python LLM application. In this quickstart we also use the Langfuse OpenAI integration to automatically capture all model parameters.

[!TIP]
Not using OpenAI? Visit our documentation to learn how to log other models and frameworks.

pip install langfuse openai

LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_BASE_URL="https://cloud.langfuse.com" # 🇪🇺 EU region
## LANGFUSE_BASE_URL="https://us.cloud.langfuse.com" # 🇺🇸 US region
## LANGFUSE_BASE_URL="https://jp.cloud.langfuse.com" # 🇯🇵 Japan region
## LANGFUSE_BASE_URL="https://hipaa.cloud.langfuse.com" # ⚕️ HIPAA region

from langfuse import observe
from langfuse.openai import openai # OpenAI integration

@observe()
def story():
    return openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "What is Langfuse?"}],
    ).choices[0].message.content

@observe()
def main():
    return story()

main()

3️⃣ See traces in Langfuse

See your language model calls and other application logic in Langfuse.

Public example trace in Langfuse

💭 Support

Finding an answer to your question:

• Our documentation is the best place to start looking for answers. It is comprehensive, and we invest significant time into maintaining it. You can also suggest edits to the docs via GitHub.
• Langfuse FAQs where the most common questions are answered.
• Use "Ask AI" to get instant answers to your questions.

Support Channels:

• Ask any question in our public Q&A on GitHub Discussions. Please include as much detail as possible (e.g. code snippets, screenshots, background information) to help us understand your question.
• Request a feature on GitHub Discussions.
• Report a Bug on GitHub Issues.
• For time-sensitive queries, ping us via the in-app chat widget.

🤝 Contributing

Your contributions are welcome!

• Vote on Ideas in GitHub Discussions.
• Raise and comment on Issues.
• Open a PR - see CONTRIBUTING.md for details on how to setup a development environment.

🥇 License

This repository is MIT licensed, except for the ee folders. See LICENSE and docs for more details.

Dependencies

We deploy this code base in Docker containers. See web/Dockerfile and worker/Dockerfile for how the images are built, including the base images they use.

⭐️ Star History

❤️ Open Source Projects Using Langfuse

Top open-source projects that use the Langfuse Python or JS/TS SDK, ranked by stars (Source):

Repository	Stars
Significant-Gravitas / AutoGPT	187903
langgenius / dify	160407
langflow-ai / langflow	156333
bytedance / deer-flow	84985
cheaper-inference / OmniRoute	84532
lobehub / lobehub	83873
abi / screenshot-to-code	81022
microsoft / ai-agents-for-beginners	78925
openai / openai-cookbook	76955
cline / cline	71312
BerriAI / litellm	62709
run-llama / llama_index	52806
LibreChat-AI / LibreChat	46745
volcengine / OpenViking	41535
HKUDS / LightRAG	40529
directus / directus	38820
DayuanJiang / next-ai-draw-io	36683
onyx-dot-app / onyx	33212
labring / FastGPT	29994
mastra-ai / mastra	29335
vercel / ai	27740
onlook-dev / onlook	27117
promptfoo / promptfoo	26836
kortix-ai / CompanyOS	20378
gastownhall / gastown	18691
camel-ai / camel	18010
suitenumerique / docs	16967
Arindam200 / awesome-ai-apps	16712
apify / apify-mcp-server	14774
The-PR-Agent / pr-agent	14686
microsoft / agent-framework	14532
The-Pocket / PocketFlow	11303
openai / codex-security	11301
tambo-ai / tambo	11222
VoltAgent / voltagent	10920
mcp-use / mcp-use	10846
yugabyte / yugabyte-db	10649
droidrun / mobilerun	9947
MervinPraison / PraisonAI	9321
miurla / morphic	9275
plastic-labs / honcho	7944
raycast / extensions	7832
2FastLabs / agent-squad	7799
xerrors / Yuxi	7601
refly-ai / refly	7547
crestalnetwork / intentkit	6524
coleam00 / ottomator-agents	5791
zenml-io / zenml	5635
colanode / colanode	5186
getzep / zep	4989
langflow-ai / openrag	4671
JoshuaC215 / agent-service-toolkit	4564
FlowElement-xinliuyuansu / m_flow	4509
panaversity / learn-agentic-ai	4395
bragai / bRAG-langchain	4196
strukto-ai / mirage	3763
awslabs / agentcore-samples	3512
ding113 / claude-code-hub	3399
rpamis / comet	3270
remix-project-org / remix-project	3096
tsedio / tsed	3091
MetaMask / metamask-mobile	3080
pingcap / autoflow	2979
wassim249 / fastapi-langgraph-agent-production-ready-template	2726
Agent-Field / agentfield	2644
suitenumerique / meet	2468
YFGaia / dify-plus	2263
PrimeIntellect-ai / prime-rl	2215
TheSpaghettiDetective / obico-server	1924
ai-hero-dev / ai-hero	1915
arabold / docs-mcp-server	1856
trailofbits / buttercup	1741
MLSysOps / MLE-agent	1580
kodustech / kodus-ai	1565
bugbasesecurity / pentest-copilot	1546
agentconnect-md / agentconnect	1514
TIGER-AI-Lab / TheoremExplainAgent	1513
hashgraph-online / awesome-codex-plugins	1508
Gsync / jobsync	1493
inkeep / agents	1490
zstar1003 / ragflow-plus	1465
hotovo / aider-desk	1463
cline / kanban	1376
wanxingai / LightAgent	1245
FranciscoMoretti / chat-js	1203
cognizant-ai-lab / neuro-san-studio	1194
okou-ai / okou	1162
alishobeiri / thread-notebook	1106
openops-cloud / openops	1081
dynamiq-ai / dynamiq	1073
trendy-design / llmchat	1072
cuga-project / cuga-agent	1070
Minima-AI-Inc / minima	1056
ZTE-AICloud / Co-Sight	1015
china-qijizhifeng / agentic-harness-engineering	959
sanity-io / next-sanity	956
plastic-labs / tutor-gpt	933
strands-agents / samples	924
andysingal / llm-course	891
groq / openbench	832

🔒 Security & Privacy

We take data security and privacy seriously. Please refer to our Security and Privacy page for more information.

Telemetry

By default, Langfuse automatically reports basic usage statistics of self-hosted instances to a centralized server (PostHog).

This helps us to:

1. Understand how Langfuse is used and improve the most relevant features.
2. Track overall usage for internal and external (e.g. fundraising) reporting.

The telemetry does not include raw traces, prompts, observations, scores, or dataset contents. We document the exact fields that are collected, where they are sent, and the implementation reference in our telemetry docs.

For Langfuse OSS, you can opt out by setting TELEMETRY_ENABLED=false.