07 Oct LangGraph Parallel Execution: Fan-Out and Fan-In

So far, our graphs ran one node at a time. But some tasks are independent: fetching facts from three sources, or summarizing five documents. Running them in parallel saves time.

The idea

              +-> fetch_a -+
START --------+            +-> combine -> END
              +-> fetch_b -+

• Fan-out: one point splits into several branches that run at the same time
• Fan-in: the branches join back together at one node

LangGraph runs in steps. All nodes that are ready in the same step run in parallel, and the graph moves on only when every one of them finishes.

Example: two parallel branches

import operator
import time
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, START, END

class State(TypedDict):
    topic: str
    results: Annotated[list[str], operator.add]
    summary: str

def fetch_a(state: State):
    time.sleep(1)                       # pretend to call a slow source
    return {"results": [f"A: notes on {state['topic']}"]}

def fetch_b(state: State):
    time.sleep(1)
    return {"results": [f"B: examples of {state['topic']}"]}

def combine(state: State):
    return {"summary": " | ".join(sorted(state["results"]))}

builder = StateGraph(State)
builder.add_node("fetch_a", fetch_a)
builder.add_node("fetch_b", fetch_b)
builder.add_node("combine", combine)

builder.add_edge(START, "fetch_a")            # fan-out: two edges from START
builder.add_edge(START, "fetch_b")
builder.add_edge(["fetch_a", "fetch_b"], "combine")   # fan-in: wait for both
builder.add_edge("combine", END)

graph = builder.compile()

start = time.time()
result = graph.invoke({"topic": "graphs"})
print(result["summary"])
print("Seconds:", round(time.time() - start, 1))

Sample output:

A: notes on graphs | B: examples of graphs
Seconds: 1.0

Both nodes slept for one second, yet the whole run took about one second, not two. That is the parallel speedup.

How it works

1. Two edges leave START, so fetch_a and fetch_b run in the same step, side by side
2. Both write to results. The reducer operator.add (Chapter 5) merges their lists instead of overwriting.
3. add_edge([“fetch_a”, “fetch_b”], “combine”) uses a list of sources, which means “run combine only after both have finished”
4. combine sees the complete merged list

Rule 1: parallel writes to one key need a reducer

If two parallel nodes write to the same key without a reducer, LangGraph doesn’t know which value wins, and raises an InvalidUpdateError saying the key can receive only one value per step.

class BadState(TypedDict):
    results: list[str]        # no reducer, so this fails when two branches write to it

The fix is the Annotated[list[str], operator.add] you already know. Alternatively, have each branch write to its own key (result_a, result_b).

Rule 2: don’t depend on the order of results

Parallel branches can finish in any order, so the order of items in results isn’t something to rely on. That is why combine uses sorted(). If order matters, sort the items, or store each result under a key that identifies its source.

Rule 3: use the list form for fan-in

There is a subtle difference:

• add_edge([“a”, “b”], “combine”): combine waits for both a and b.
• add_edge(“a”, “combine”) plus add_edge(“b”, “combine”): combine runs whenever a step finishes with a or b complete, which can be more than once if the branches have different lengths.

When branches take different numbers of steps, always use the list form for a clean join.

Dynamic fan-out with Send

The example above has exactly two branches, decided when you wrote the code. What if the number depends on the data? For example, “process every document in this list.” That is the map-reduce pattern, and Send handles it.

A routing function returns a list of Send(node_name, input) objects, one per item:

import operator
from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.types import Send

class State(TypedDict):
    words: list[str]
    lengths: Annotated[list[str], operator.add]
    report: str

def fan_out(state: State):
    return [Send("measure", {"word": w}) for w in state["words"]]

def measure(payload: dict):
    word = payload["word"]
    return {"lengths": [f"{word}={len(word)}"]}

def report(state: State):
    return {"report": ", ".join(sorted(state["lengths"]))}

builder = StateGraph(State)
builder.add_node("measure", measure)
builder.add_node("report", report)

builder.add_conditional_edges(START, fan_out, ["measure"])
builder.add_edge("measure", "report")
builder.add_edge("report", END)

graph = builder.compile()

print(graph.invoke({"words": ["graph", "node", "edge"]})["report"])

Output:

edge=4, graph=5, node=4

How it works:

1. fan_out returns three Send objects, one per word. The number of branches was decided at runtime.
2. Each Send runs measure with its own input ({“word”: …}), so the node receives that payload, not the whole state
3. Each run writes to lengths, and the reducer merges all the results
4. When all branches finish, report runs once with the merged list
5. Pass a list of node names ([“measure”]) as the third argument to add_conditional_edges so LangGraph knows where the Send calls can go

Send 3 words, get 3 parallel runs. Send 300 words, and you get 300.

A realistic use with an LLM

The same shape works for real AI tasks:

• Summarize each of several documents in parallel, then combine the summaries
• Ask the model to write one short explanation per topic, then assemble a study guide
• Run several search tools at once and merge the findings

Just put llm.invoke(…) inside the parallel node. A Studyopedia-style example: one node per chapter topic, each generating a quiz question, then a final node that collects them all.

Controlling how much runs at once

Too many parallel model calls can hit your provider’s rate limits and cost more at once. Limit concurrency in the config:

graph.invoke({"words": ["a", "b", "c", "d"]}, {"max_concurrency": 2})

At most two branches run at the same time, and the rest wait their turn.

When something fails

A step is all-or-nothing: if one parallel branch raises an error, the step fails and the run stops with that error. Handle expected errors (like a missing page) inside the node, and return a friendly result such as “Source unavailable” instead of crashing everything.

Async note

For many network calls, async nodes (async def with await) plus ainvoke() or astream() use resources more efficiently than threads. The graph structure stays the same.

Common mistakes

• No reducer on a shared key: parallel writes to the same key without a reducer raise an error
• Relying on result order: sort, or key results by source
• Two separate add_edge calls to the join node when branches differ in length: use the list form
• Forgetting the third argument in add_conditional_edges when using Send: pass the list of target nodes
• Reading the whole state inside a Send node: it receives only the payload you sent
• Launching hundreds of LLM calls at once: set max_concurrency and watch costs
• Depending on one branch’s output inside another: parallel nodes can’t see each other’s results in the same step

Summary

• Nodes ready in the same step run in parallel
• Fan-out: several edges from one node. Fan-in: add_edge([…], “join”) waits for all.
• Parallel writes to one key need a reducer, and results may arrive in any order
• Send creates a dynamic number of parallel branches for map-reduce
• Use max_concurrency to respect rate limits and cost

Practice

1. Add a third branch, fetch_c, to the first example and confirm the run still takes about one second
2. Remove the reducer from results and read the error message
3. Change the Send example to measure 10 words, and print the report
4. Set max_concurrency to 1 in the timing example and compare the total seconds

---

If you liked the tutorial, spread the word and share the link and our website, Studyopedia, with others.

---

For Videos, Join Our YouTube Channel: Join Now

---

Read More:

• Generative AI Tutorial
• AI Ethics
• Machine Learning Tutorial
• Deep Learning Tutorial
• Ollama Tutorial
• Retrieval Augmented Generation (RAG) Tutorial
• ChatGPT Tutorial
• Microsoft Copilot Tutorial