07 Oct Add an LLM Node in LangGraph: Build Your First Chatbot

Until now, our nodes ran plain Python. Now we will add an LLM node, which is just a normal node that calls a language model. The graph itself doesn’t change.

Before you start

You need:

• Your .env file with an API key (Chapter 3)
• One model package installed: langchain-openai or langchain-anthropic

Step 1: Create the model

from dotenv import load_dotenv
load_dotenv()   # loads the API key from .env

# Option A: Anthropic
from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(model="claude-sonnet-5-5")

# Option B: OpenAI
# from langchain_openai import ChatOpenAI
# llm = ChatOpenAI(model="gpt-4o-mini")

Use one option only. Model names change over time, so check your provider’s model list if you get a “model not found” error.

Step 2: Test the model on its own

response = llm.invoke("Explain what a variable is in one sentence.")
print(response.content)

Sample output (yours will differ):

A variable is a named container that stores a value your program can use and change.

invoke() returns a message object. The text is in .content.

Step 3: Put the LLM inside a node

from typing import TypedDict
from langgraph.graph import StateGraph, START, END

class State(TypedDict):
    question: str
    answer: str

def ask_llm(state: State):
    response = llm.invoke(state["question"])
    return {"answer": response.content}

builder = StateGraph(State)
builder.add_node("ask_llm", ask_llm)
builder.add_edge(START, "ask_llm")
builder.add_edge("ask_llm", END)

graph = builder.compile()

result = graph.invoke({"question": "What is Python? Answer in two sentences."})
print(result["answer"])

Sample output:

Python is a popular, beginner-friendly programming language known for its clear, readable syntax. It is used for web development, data analysis, automation, and AI.

How it works:

1. The state holds a question and an empty answer
2. ask_llm sends the question to the model
3. The model’s reply is saved into answer
4. The graph ends

It is the same pattern as Chapter 4, with an LLM call inside the node.

Step 4: Give the model a role with a system prompt

A system prompt tells the model how to behave. Pass messages as (role, text) pairs:

def ask_llm(state: State):
    messages = [
        ("system", "You are a friendly tutor. Explain things simply for beginners, in under 60 words."),
        ("human", state["question"]),
    ]
    response = llm.invoke(messages)
    return {"answer": response.content}

Run the graph again and the answers will be shorter and simpler, because the model now follows the tutor instructions.

Step 5: A two-node LLM graph

Real apps split the work into steps. Here, one node builds a better prompt and the next calls the model:

class State(TypedDict):
    topic: str
    level: str
    prompt: str
    answer: str

def build_prompt(state: State):
    prompt = f"Explain '{state['topic']}' to a {state['level']} in 3 short sentences."
    return {"prompt": prompt}

def ask_llm(state: State):
    response = llm.invoke(state["prompt"])
    return {"answer": response.content}

builder = StateGraph(State)
builder.add_node("build_prompt", build_prompt)
builder.add_node("ask_llm", ask_llm)

builder.add_edge(START, "build_prompt")
builder.add_edge("build_prompt", "ask_llm")
builder.add_edge("ask_llm", END)

graph = builder.compile()

result = graph.invoke({"topic": "recursion", "level": "school student"})
print(result["answer"])

Change level to “complete beginner” or “college student” and the explanation adapts. Keeping prompt-building in its own node makes it easy to test and change later.

Important limitation

This chatbot has no memory. Each invoke() is a fresh start, so if you ask a follow-up like “Can you give an example?”, the model has no idea what you were discussing. Chapter 9 fixes this with MessagesState.

Common mistakes

• Forgetting load_dotenv(): you will get an authentication error about a missing key
• Returning the whole response object: use response.content to store just the text
• Wrong model name: check your provider’s current list
• Installing one package but importing the other: langchain-openai goes with ChatOpenAI, and langchain-anthropic goes with ChatAnthropic
• Forgetting that calls cost money: LLM calls are billed by usage, so avoid running them inside a loop without a limit (Chapter 7)

Summary

• An LLM node is a normal node that calls llm.invoke() and returns response.content
• A system prompt shapes how the model responds
• Splitting prompt-building and model-calling into separate nodes keeps graphs clean
• This chatbot answers one question at a time and has no memory yet

Practice

1. Change the system prompt so the tutor answers only in bullet points
2. Add a third node, shorten, that trims the answer to its first sentence (hint: answer.split(“.”)[0])
3. Add a language field to the state and ask the model to reply in that language

If you liked the tutorial, spread the word and share the link and our website, Studyopedia, with others.

For Videos, Join Our YouTube Channel: Join Now

Read More:

• Generative AI Tutorial
• AI Ethics
• Machine Learning Tutorial
• Deep Learning Tutorial
• Ollama Tutorial
• Retrieval Augmented Generation (RAG) Tutorial
• ChatGPT Tutorial
• Microsoft Copilot Tutorial