07 Oct Deploy a LangGraph App: LangSmith Deployment and FastAPI

Your graph works on your laptop. To let real users reach it, you need to deploy it: run it on a server, protect it, and keep its memory in a real database. This chapter covers the two main routes and a production checklist.

Update to our outline: this chapter was planned around “LangGraph Platform.” That product now falls under LangSmith Deployment, and the server that runs your graphs is called the Agent Server. You will see the older name in older articles.

Two routes

• Route A: LangSmith Deployment (LangChain’s runtime). You manage only your code. It is the fastest path to production, with streaming, threads, and memory built in.
• Route B: Your own API (FastAPI). You manage the server, database, security, and scaling. It is good for full control, and for fitting into an existing website or backend.

Both run the same graph you already built.

Route A: LangSmith Deployment

LangSmith Deployment runs the same Agent Server in several hosting models:

• Cloud: fully managed by LangChain on AWS and GCP, and requires a Plus plan or above
• Standalone server: you run the containers yourself (Docker, Compose, or Kubernetes) with your own PostgreSQL, Redis, and a LangSmith license
• Self-hosted: you run the full LangSmith platform in your own cloud, and it requires an Enterprise plan

Plans and pricing change, so check the current LangSmith pricing page before choosing.

The workflow

1. Test locally with langgraph dev and Studio (Chapter 20)
2. Make sure your project has a langgraph.json pointing at your compiled graph, plus your dependencies listed (pyproject.toml or requirements.txt)
3. Push your code to GitHub (with .env in .gitignore)
4. Create a deployment from the LangSmith web UI by connecting the repository, or use the langgraph deploy command
5. Add your secrets (API keys) in the deployment’s settings, and never in your code
6. Call your graph over HTTP, using the SDK or plain API calls

Building the container yourself

For standalone or self-hosted routes, the CLI builds a Docker image from your project:

langgraph build -t my-support-bot

The self-hosted workflow is: test locally with langgraph dev, build a Docker image with langgraph build, push it to a container registry your infrastructure can reach, then deploy from the control plane UI.

What you get

• Persistence (checkpoints, threads, and a store) handled by the server’s database, so you don’t wire up checkpointers yourself
• Streaming, human-in-the-loop pauses, and background runs
• Studio connected to the deployment for debugging (Chapter 20)
• Scaling without you managing servers, on the managed option

Chapters 13 to 16 map directly onto this: thread_id, interrupt(), Command(resume=…), and stream_mode are the same ideas, exposed through an API.

Route B: Your own FastAPI service

If you want your graph inside your own backend, wrap it in a web API. Here is a minimal version, using a chatbot with memory (Chapter 13).

pip install -U fastapi "uvicorn[standard]"

app.py

import json
import re
from dotenv import load_dotenv
from fastapi import Depends, FastAPI, Header, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import SystemMessage
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph, MessagesState, START, END

load_dotenv()
llm = ChatAnthropic(model="claude-sonnet-5-5")

def as_text(content):
    if isinstance(content, str):
        return content
    return "".join(b.get("text", "") for b in content if isinstance(b, dict))

# ---- the graph (Chapter 13) ----
async def chatbot(state: MessagesState):
    system = SystemMessage("You are a friendly tutor. Keep answers under 80 words.")
    reply = await llm.ainvoke([system] + state["messages"])
    return {"messages": [reply]}

builder = StateGraph(MessagesState)
builder.add_node("chatbot", chatbot)
builder.add_edge(START, "chatbot")
builder.add_edge("chatbot", END)
graph = builder.compile(checkpointer=InMemorySaver())   # swap for a database in production

# ---- the API ----
app = FastAPI()

API_KEYS = {"demo-key-123": "user-1"}      # DEMO ONLY: use real authentication

def current_user(x_api_key: str = Header(default="")) -> str:
    user = API_KEYS.get(x_api_key)
    if user is None:
        raise HTTPException(status_code=401, detail="Invalid API key")
    return user

class ChatRequest(BaseModel):
    message: str = Field(min_length=1, max_length=2000)
    chat_id: str = Field(default="main", pattern=r"^[a-zA-Z0-9_-]{1,40}$")

def make_config(user: str, chat_id: str) -> dict:
    # The USER part comes from the server, never from the client
    return {"configurable": {"thread_id": f"{user}-{chat_id}"}, "recursion_limit": 10}

@app.post("/chat")
async def chat(req: ChatRequest, user: str = Depends(current_user)):
    result = await graph.ainvoke(
        {"messages": [("human", req.message)]},
        make_config(user, req.chat_id),
    )
    return {"reply": as_text(result["messages"][-1].content)}

@app.post("/chat/stream")
async def chat_stream(req: ChatRequest, user: str = Depends(current_user)):
    config = make_config(user, req.chat_id)

    async def events():
        async for chunk, meta in graph.astream(
            {"messages": [("human", req.message)]}, config, stream_mode="messages"
        ):
            if meta.get("langgraph_node") == "chatbot":
                text = as_text(chunk.content)
                if text:
                    yield f"data: {json.dumps({'token': text})}\n\n"
        yield "data: [DONE]\n\n"

    return StreamingResponse(events(), media_type="text/event-stream")

@app.get("/health")
async def health():
    return {"status": "ok"}

Run and test

uvicorn app:app --reload

curl -X POST http://127.0.0.1:8000/chat \
  -H "Content-Type: application/json" \
  -H "X-API-Key: demo-key-123" \
  -d '{"message": "My name is Asha", "chat_id": "c1"}'

Sample output:

{"reply":"Nice to meet you, Asha! What would you like to learn today?"}

Send a second request with the same chat_id, asking “What’s my name?”, and the bot remembers, thanks to the thread. FastAPI also generates an interactive test page at http://127.0.0.1:8000/docs.

What this code gets right

• The user comes from the server. The thread ID is built from an authenticated user plus a client-chosen chat_id, so one user can’t read another’s conversation by guessing IDs (Chapter 14).
• Inputs are validated. Message length is capped, and chat_id must match a safe pattern.
• Async all the way: ainvoke, astream, and await llm.ainvoke let the server handle many users at once (Chapter 16).
• A recursion limit protects against runaway loops (Chapter 7).
• Streaming sends tokens as server-sent events.

What is deliberately simplified

• API_KEYS is a demo. Use real authentication (OAuth, sessions, or your framework’s auth) and per-user rate limits.
• InMemorySaver loses everything on restart, and also breaks when you run multiple server processes, because each keeps its own memory. Use a database checkpointer.

Moving memory to Postgres

pip install -U langgraph-checkpoint-postgres "psycopg[binary,pool]"

from contextlib import asynccontextmanager
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver

DB_URI = "postgresql://user:password@localhost:5432/mydb"   # from an environment variable in real use

@asynccontextmanager
async def lifespan(app: FastAPI):
    async with AsyncPostgresSaver.from_conn_string(DB_URI) as checkpointer:
        await checkpointer.setup()                 # creates tables the first time
        app.state.graph = builder.compile(checkpointer=checkpointer)
        yield

app = FastAPI(lifespan=lifespan)

Your endpoints then use app.state.graph instead of the global graph. Setup details for checkpointer packages can change, so confirm them in the current LangGraph persistence docs.

Handling human-in-the-loop over an API

With a web API, an interrupt() (Chapter 15) becomes two requests:

1. /chat returns the pause. Check “__interrupt__” in result, and send the question to the client.
2. A second endpoint, such as /resume, receives the answer and calls graph.ainvoke(Command(resume=answer), config) with the same thread_id.

This is why a persistent checkpointer matters: the pause may last hours, and your server may restart in between.

Production checklist

Secrets and configuration:

• API keys in environment variables or a secrets manager, never in code or Git
• .env files kept out of Docker images and repositories

Security:

• HTTPS only
• Real authentication, with user IDs taken from the login, never from request bodies
• Rate limits per user, and a cap on message length
• CORS limited to your own website’s origin
• Treat model output and tool results as untrusted text, and validate before acting on them
• Human approval for risky tools (Chapter 15)

Cost and reliability:

• recursion_limit on every run, and max_concurrency for parallel graphs
• Timeouts on tool and web requests
• A monthly budget alert with your model provider
• A /health endpoint, so your host can restart a stuck server
• Handle model errors gracefully, and return a friendly message instead of a stack trace

Data:

• Persistent database with backups
• A plan to delete old conversations, and a way for users to delete their data
• Logs that don’t include full private messages or secrets

Quality:

• Tests for your tools and routers (Chapter 21)
• Tracing in LangSmith to review real conversations (Chapter 20), respecting user privacy
• Pinned dependency versions in requirements.txt, since these libraries change quickly

Choosing a route

• You want production fast, with streaming, threads, and Studio included: LangSmith Deployment
• You must keep everything on your own servers: standalone server or self-hosting
• You already have a backend and want full control: FastAPI (Route B)
• A small internal tool with a handful of users: FastAPI with SQLite or Postgres

You can start with FastAPI and move to LangSmith Deployment later, since your graph code stays the same. Only the wrapping changes.

Common mistakes

• Building the thread ID from client input alone: prefix it with a server-verified user ID
• Using InMemorySaver in production: memory disappears on restart and isn’t shared between processes
• Calling a sync graph method from an async endpoint: use ainvoke and astream
• Committing .env or baking it into a Docker image: inject secrets at runtime
• No rate limits: one user or bot can run up your model bill
• Returning raw errors to users: log details privately, and send a simple message
• Exposing debug or docs pages publicly: disable or protect them in production
• Skipping load testing: try several simultaneous users before launch

Summary

• LangSmith Deployment (formerly LangGraph Platform) runs your graph on an Agent Server, in managed cloud, standalone containers, or self-hosted
• The CLI flow is langgraph dev to test, then langgraph build for a container image
• Or wrap the graph in FastAPI, with server-side auth, validated inputs, streaming, and a database checkpointer
• Pauses and approvals become two requests that share a thread_id
• Use the checklist: secrets, auth, limits, persistence, testing, and privacy

Practice

1. Run the FastAPI app, and send two requests on the same chat_id to confirm memory works
2. Send a request with a wrong API key, and one with an empty message, and read the errors
3. Test the streaming endpoint with curl -N and the same headers, and watch the tokens arrive
4. Add a /resume endpoint for the refund flow from Chapter 21, and think through what should be checked before approving

---

If you liked the tutorial, spread the word and share the link and our website, Studyopedia, with others.

---

For Videos, Join Our YouTube Channel: Join Now

---

Read More:

• Generative AI Tutorial
• AI Ethics
• Machine Learning Tutorial
• Deep Learning Tutorial
• Ollama Tutorial
• Retrieval Augmented Generation (RAG) Tutorial
• ChatGPT Tutorial
• Microsoft Copilot Tutorial