Skip to content
AI & Tools

LLM Observability Tools Compared: Langfuse, Helicone, W&B Weave and LangSmith

AIDToolStack EditorialNovember 7, 20266 min read
LLM ObservabilityLangfuseHeliconeLangSmithWeave
Share:Twitter

The first time an LLM feature misbehaves in production, you discover that your normal logs are useless. You can see that /api/chat returned 200 in 4.1 seconds. You can't see the prompt that was actually sent after the template filled in, which retrieved documents went with it, what the model said, what it cost, or which of the three tool calls went sideways. LLM observability tools exist to record exactly that.

This post compares four of them as of November 2026: Langfuse, Helicone, Weights & Biases Weave and LangSmith. Two of the four changed owners in the past year, and one of those changes affects whether I'd start a new project on it. If you only need plain request logs for a small app, our post on logging and observability for small applications may be enough.

Two ways to collect the data

Before comparing features, decide how you want data to leave your app. There are two models, and they shape everything else.

Proxy versus SDK instrumentation: a gateway sees every request it forwards, while an SDK records spans inside your code and ships them separately
Proxy versus SDK instrumentation: a gateway sees every request it forwards, while an SDK records spans inside your code and ships them separately

Proxy (gateway). Your app sends model calls to the tool's URL instead of the provider's. The tool forwards them and logs each request on the way. Helicone works this way. Its quick start is literally a base URL change on the OpenAI client:

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://ai-gateway.helicone.ai",
    api_key=os.getenv("HELICONE_API_KEY"),
)

That's the lowest-effort setup there is, and the same page says the gateway gives you one OpenAI-compatible API for 100+ models, so switching providers is a change of model name. The cost: the proxy sits in the request path, and it only sees model calls. Your retrieval step, your tool code and your business logic are invisible to it.

SDK (tracing). You instrument your code and the SDK records nested spans (a request, the retrieval inside it, the generation inside that), then ships them to the backend in the background. Langfuse, Weave and LangSmith work this way. More setup, but you get the whole tree. Langfuse's get-started page offers a drop-in OpenAI wrapper so the first step is small:

# pip install langfuse; set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL
from langfuse.openai import openai

Weave's README starts with weave.init("<your-team>/<your-project-name>") and traces your own functions with the @weave.op decorator; for the OpenAI Agents SDK, Claude Agent SDK and Google ADK it says weave.init() is enough.

If you're debugging agents, I'd pick an SDK-based tool. A proxy can tell you the fourth model call was slow; it can't tell you that the fourth call happened because the retriever returned an empty list.

Status check: who owns what in November 2026

ToolLicenceOwner newsSelf-hosting
LangfuseMIT, except the ee folders (GitHub)Joined ClickHouse on January 16, 2026; "No licensing changes planned" (announcement)Free; Docker Compose to start, Helm for production
HeliconeApache 2.0 (GitHub)Acquired by Mintlify on March 3, 2026 and put in maintenance mode (Mintlify); pricing page now says it's "no longer part of Mintlify"Docker Compose; Helm chart for enterprise
W&B WeaveWeave SDK is Apache-2.0 (GitHub)W&B is part of CoreWeave; pricing now lives on CoreWeave's siteDedicated deployment on Enterprise
LangSmithNot stated on the pages I usedLangChainEnterprise add-on only (docs)

The Helicone row needs a sentence of its own. Mintlify's March post said Helicone "will continue operating in maintenance mode" with security updates, bug fixes and new models, and that Mintlify would help customers migrate to another platform. When I loaded Helicone's pricing page today, a banner read "Helicone is no longer part of Mintlify. Details to come soon." I couldn't find any detail beyond that banner. Until there's a public roadmap, I wouldn't start a new project on Helicone; existing users don't need to panic, since it's Apache-licensed and self-hostable.

Langfuse's move is less worrying. Its own post says Cloud "keeps running as-is" and self-hosting stays "a first-class path."

What self-hosting actually involves

"Open source and self-hostable" covers a wide range of effort, so it's worth knowing what you'd be running.

Langfuse is the easiest to try. The README has you clone the repo and run docker compose up; for production it recommends Kubernetes via Helm and ships Terraform templates for AWS, Azure and GCP. The ee folders are outside the MIT licence, so check which features you need before assuming the self-hosted build has everything Cloud has.

Helicone also starts with Docker Compose (./helicone-compose.sh helicone up per its README), but the production Helm chart is for enterprise workloads and you have to email the company for access.

LangSmith is a different class of project. Self-hosting is "an add-on to the Enterprise plan" and needs a licence key from sales. The self-hosting docs list the pieces: a frontend, backend, queue, platform backend, Playground and a code-execution backend, plus ClickHouse for traces, PostgreSQL for operational data, Redis or Valkey for queues and caching, and blob storage. Everything is bundled by default, but the docs recommend external storage in production. That's a job for whoever runs your Kubernetes clusters, not a weekend experiment.

Weave self-hosting sits under CoreWeave's Enterprise tier as a dedicated deployment, with options like customer-managed encryption keys and bring-your-own bucket.

Pricing as of November 2026

Each tool bills a different unit, which makes direct comparison awkward. Langfuse counts "units", Helicone counts requests, Weave counts megabytes ingested, and LangSmith counts traces.

ToolFree tierFirst paid planOverage / notes
Langfuse (pricing)Hobby: 50k units/month, 30 days of data, 2 usersCore, $29/month: 100k units, 90 days$8 per 100k units, dropping to $6 above 50M
Helicone (pricing)Hobby: 10,000 requests, 7-day retentionPro, $79/month: 1-month retentionUsage-based past 10k requests
W&B Weave (pricing)1 GB ingestion/monthPro, from $60/month: 1.5 GB/month$0.10/MB on Pro
LangSmith (pricing)Developer: 1 seat, 5k base traces/monthPlus, $39/seat/month: 10k base tracesPay as you go; base traces kept 14 days, extended 180

Two things the table hides. LangSmith charges per seat on Plus, so a team of eight pays for eight seats before any traces. Weave charges by data volume, so long prompts and big retrieved contexts cost more than short ones even at the same request count. Langfuse's Core plan has unlimited users, which is why it often ends up cheapest for small teams.

Picking one

  • You want open source you can run yourself, today, for free: Langfuse. MIT core, Docker Compose to start, Helm for production. Our Langfuse vs Helicone comparison goes deeper.
  • Your team already uses W&B for model training: Weave keeps experiments and LLM traces in one place. See W&B vs Langfuse.
  • You build on LangChain or LangGraph and won't self-host: LangSmith is the native choice; budget for seats.
  • You want one gateway in front of many providers more than deep traces: Helicone's design fits, but weigh its unclear ownership first.

Whatever you choose, decide early what you must not log. Prompts contain user data, and these tools exist to capture prompts and outputs. Check each one's masking and retention options before the first production trace, not after the first privacy review. For the calling side of the same problem, see our LLM API best practices guide.

AE

Written by AIDToolStack Editorial

Researched from official documentation, specifications and vendor pages, listed under Sources below. Last checked November 2026.

Sources

  1. docs.helicone.ai/getting-started/quick-start
  2. langfuse.com/docs/observability/get-started
  3. github.com/wandb/weave
  4. github.com/langfuse/langfuse
  5. langfuse.com/blog/joining-clickhouse
  6. github.com/Helicone/helicone
  7. www.mintlify.com/blog/mintlify-acquires-helicone
  8. docs.langchain.com/langsmith/self-hosted
  9. www.helicone.ai/pricing
  10. langfuse.com/pricing
  11. coreweave.com/forge-pricing
  12. www.langchain.com/pricing

Related Posts

Explore on AIDToolStack

Discover tools and servers featured in our directory.

MCP Servers

Trigger.dev Jobs MCP ServerManage Trigger.dev background jobs from your AI coding assistant. Monitor job runs, trigger tasks manually, inspect execution logs, manage schedules, and debug failed runs. The server provides operational visibility into your background job infrastructure — see which tasks are running, queued, or failed, and take action. Your AI can help diagnose job failures from execution logs, retry failed tasks, and optimize concurrency configurations for better throughput.Tailscale Network MCP ServerManage your Tailscale mesh VPN network from your AI coding assistant. View connected devices, manage ACLs, configure exit nodes, check device status, and manage DNS settings. The server provides visibility into your Tailscale network — see which machines are online, troubleshoot connectivity issues, manage network access policies, and configure routes. Useful for developers managing distributed infrastructure where Tailscale provides secure networking between services.Portainer Docker MCP ServerManage Docker environments through Portainer's API from your AI coding assistant. Control containers, images, networks, and volumes across multiple Docker hosts and Kubernetes clusters. The server provides tools for container lifecycle management, image pulling and building, stack deployment, and resource monitoring. Portainer's multi-environment management makes this server particularly valuable for teams managing containers across development, staging, and production environments.

Reviews

Sign in to leave a review.

No reviews yet. Be the first!

Comments

Sign in to leave a comment.

No comments yet. Be the first!

Sponsor Our Newsletter

Reach thousands of developers who are actively evaluating AI tools, MCP servers, and dev infrastructure. Our weekly newsletter goes to engaged technical decision-makers.

Growing subscriber baseHigh open rates

All sponsored content is clearly labeled per our editorial policy.