Running MCP Servers in Production: Lessons from Our First Six Months

When we launched our MCP (Model Context Protocol) server registry, I thought we understood the protocol well. We had read the spec, built test servers, integrated with Claude and other clients, and documented everything. Six months of production traffic proved that understanding the spec and understanding real-world usage are very different things.

Here is what surprised us and what we learned running MCP infrastructure at scale.

The Protocol Is Simple; Implementations Are Not

The MCP spec defines a clean, JSON-RPC-based protocol for AI models to connect with external tools and data sources. Read the spec and it feels straightforward: tools, resources, prompts, transport layers. Implement it in production and you discover that every MCP server has different interpretations of error handling, timeout behavior, authentication flows, and capability declarations.

Some servers return structured errors. Others return raw strings. Some handle authentication through OAuth flows. Others expect API keys in headers. Some declare their capabilities accurately in the manifest. Others claim capabilities they partially support.

Our registry had to normalize all of this. We built a compatibility layer that tests each server against our conformance suite: does it handle invalid inputs gracefully? Does authentication actually work? Do declared tools match actual behavior? Does it respond within acceptable timeouts?

Roughly 30% of servers we initially listed failed our conformance tests. Not because they were broken — they worked fine with specific clients — but because they made assumptions about how they would be called that did not hold universally.

Cold Start Is the Real Problem

Most MCP servers are deployed as serverless functions or containerized services that scale to zero when idle. When an AI model tries to use a tool, the server needs to be available within seconds. But cold starts on container services can take 5-30 seconds depending on the runtime and dependencies.

This matters because AI tool-calling is interactive. A user asks Claude to "check the weather in Tokyo" and expects a response within 2-3 seconds. If the weather MCP server has a 10-second cold start, the entire interaction feels broken.

Our solutions: we maintain a health-check ping every 60 seconds for popular servers to keep them warm. We display estimated response time in our registry so developers know what to expect. We flag servers with consistently high latency.

The best-performing MCP servers we have seen use lightweight runtimes (Bun, Deno, or compiled Go binaries) that cold-start in under 2 seconds. Python servers with large dependency trees (TensorFlow, pandas) consistently have the worst cold-start performance.

Authentication Is a UX Disaster

The MCP spec supports multiple authentication patterns, but in practice, authentication is where most user frustration originates.

The flow typically requires: user configures the MCP server in their AI client, the client attempts to connect, the server requests authentication, the user must somehow provide credentials (API key, OAuth token, etc.), the connection is established.

The problem: every client handles this differently. Some open a browser window for OAuth. Some ask for an API key in a text field. Some read from environment variables. There is no standard UX for MCP authentication, so every server-client combination feels different.

What we recommend to MCP server developers: support API key authentication (simplest to implement, works everywhere). If you must use OAuth, provide a setup page that generates a long-lived token the user can paste into their client. Do not require interactive OAuth flows — they break in headless environments and confuse users.

Error Messages Matter More Than You Think

When an MCP tool call fails, the error message goes back to the AI model, which interprets it and presents a response to the user. Bad error messages create bad user experiences.

Example of a bad error: "Error 500: Internal Server Error." The AI model has nothing useful to work with. It tells the user "something went wrong" without any actionable guidance.

Example of a good error: "Rate limit exceeded. This API allows 10 requests per minute. Please wait 45 seconds before retrying." The AI model can communicate this clearly to the user and even suggest waiting or trying an alternative approach.

We added error message quality to our conformance testing. Servers that return generic errors get flagged in our registry with a note about error handling quality.

The Discovery Problem

There are now hundreds of MCP servers available. How does a developer find the right one for their use case? How does an AI model know which server to call for a given task?

We solve part of this with our registry — categorization, search, reviews, and compatibility information. But the deeper problem is semantic: when a user says "book me a flight," how does the AI model know which MCP server handles flight bookings? There might be three different ones with different APIs, different airlines supported, and different pricing models.

This is where tool descriptions matter enormously. The description field in an MCP tool declaration is not documentation for humans — it is the primary signal for AI models deciding whether to use that tool. Well-written descriptions that clearly state what the tool does, what inputs it needs, and what limitations it has dramatically improve tool selection accuracy.

We score and display description quality in our registry. Servers with vague descriptions ("does stuff with files") rank lower than servers with precise descriptions ("reads, writes, and lists files in a specified directory with support for glob patterns and recursive operations").

Versioning and Breaking Changes

MCP servers evolve. Tools get renamed, parameters change, response formats update. When a server pushes a breaking change, every client that depends on it breaks.

There is no built-in versioning mechanism in the MCP protocol (as of the current spec). Server developers must handle versioning themselves — through URL paths (/v1/, /v2/), capability negotiation, or maintaining backwards compatibility indefinitely.

Our recommendation: treat MCP server APIs like public APIs. Maintain backwards compatibility. Use additive changes (new optional parameters, new tools) rather than breaking changes. When breaking changes are unavoidable, support the old version for at least 30 days and communicate the deprecation clearly.

We track version history in our registry and alert users when servers they depend on push breaking changes.

What Makes a Great MCP Server

After evaluating hundreds of servers, patterns emerge:

Fast cold start (under 3 seconds): use lightweight runtimes, minimize dependencies.

Clear tool descriptions: tell the AI model exactly what each tool does, including limitations.

Structured errors: return actionable error messages that help the AI model recover or inform the user.

Simple authentication: API key-based auth that works everywhere without complex OAuth flows.

Predictable behavior: same inputs always produce same outputs. No surprising side effects.

Good documentation: README with usage examples, parameter descriptions, and common patterns.

The MCP ecosystem is still young. Standards and best practices are still forming. But the servers that excel on these dimensions today are the ones that will remain relevant as the ecosystem matures and competition increases.

Our Roadmap

Based on six months of production experience, we are building: automated compatibility testing against popular AI clients (not just spec compliance), performance monitoring and latency tracking, a review system where developers rate servers based on real usage, and better search that uses AI to match user intent with server capabilities.

The MCP ecosystem has enormous potential. It is the closest thing we have to a universal plugin system for AI models. But it needs better tooling, better standards, and better discovery. That is what we are building.