04 Oct Structured Output Guardrails: Validation with Pydantic

Many applications do not just show the AI’s reply to a person. They feed it into a program. For example, a program may expect a recipe with a name, a cooking time, and a list of ingredients. If the model replies with a friendly sentence instead, or gives a cooking time of “a long time”, the program can crash or behave in unexpected ways.

A format guardrail solves this problem by checking that the reply has the right structure and the right types of values. In this chapter, you will learn how to do this with Pydantic, a popular Python library for data validation.

What is Pydantic?

Pydantic lets you describe the shape of your data as a Python class. Then it checks incoming data against that description and tells you exactly what is wrong. You installed Pydantic in Chapter 4. This chapter uses Pydantic version 2.

Example 1: Describing the Data

We describe a recipe by writing a class that inherits from BaseModel. Each line inside the class is a field with a name and a type.

from pydantic import BaseModel

class Recipe(BaseModel):
    name: str
    cooking_time_minutes: int
    ingredients: list[str]

data = {
    "name": "Pasta",
    "cooking_time_minutes": 10,
    "ingredients": ["pasta", "salt", "water"],
}

recipe = Recipe.model_validate(data)

print(recipe.name)
print(recipe.cooking_time_minutes)
print(recipe)

Output

Pasta
10
name='Pasta' cooking_time_minutes=10 ingredients=['pasta', 'salt', 'water']

Understanding the Code

• class Recipe(BaseModel) defines the structure we expect.
• name: str means the name must be text. cooking_time_minutes: int means the time must be a whole number. ingredients: list[str] means a list of text values.
• Recipe.model_validate(data) checks the dictionary against the class. If the data is valid, it returns a Recipe object.
• You can then read the values safely using recipe.name and recipe.cooking_time_minutes.

One thing to know: by default, Pydantic tries to convert values when it can. For example, the text “10” is converted into the number 10. That is usually convenient, but remember it when you are testing.

Example 2: Validating the Model’s JSON Reply

AI models usually return structured data as JSON text. Pydantic can read JSON text directly with model_validate_json. If the reply is not valid, Pydantic raises a ValidationError, and we can catch it.

from pydantic import BaseModel, ValidationError

class Recipe(BaseModel):
    name: str
    cooking_time_minutes: int
    ingredients: list[str]

replies = [
    '{"name": "Pasta", "cooking_time_minutes": 10, "ingredients": ["pasta", "salt"]}',
    '{"name": "Pasta", "cooking_time_minutes": "a long time", "ingredients": "pasta"}',
    "Here is your recipe: Pasta",
]

for reply in replies:
    try:
        recipe = Recipe.model_validate_json(reply)
        print("Valid reply:", recipe.name)
    except ValidationError as error:
        print("Invalid reply:")
        for problem in error.errors():
            field = ".".join(str(part) for part in problem["loc"]) or "(whole reply)"
            print("  -", field, ":", problem["type"])

Output

Valid reply: Pasta
Invalid reply:
  - cooking_time_minutes : int_parsing
  - ingredients : list_type
Invalid reply:
  - (whole reply) : json_invalid

Understanding the Code

• The first reply is correct JSON with the right types, so it passes.
• The second reply is valid JSON, but the cooking time is not a number and the ingredients are not a list. Pydantic reports both problems.
• The third reply is not JSON at all, which is what happens when a model adds friendly text instead of data.
• error.errors() returns a list of problems. Each problem has a “loc” (the location, which is the field name) and a “type” (a short code that describes what went wrong).
• The type codes int_parsing, list_type, and json_invalid mean “could not read as a whole number”, “not a list”, and “not valid JSON”.

Adding Rules to Fields

Types alone are not enough. A cooking time of 0 or -5 minutes has the right type but makes no sense. An empty ingredient list is also suspicious. Pydantic lets you add rules with Field, and you can write your own rules with a field validator.

• min_length and max_length limit the length of text or lists.
• gt, ge, lt, le mean greater than, greater than or equal, less than, and less than or equal. They limit numbers.
• field_validator lets you write your own check as a small function.

Example 3: Constraints and a Custom Validator

from pydantic import BaseModel, Field, ValidationError, field_validator

class Recipe(BaseModel):
    name: str = Field(min_length=1, max_length=50)
    cooking_time_minutes: int = Field(gt=0, le=480)
    ingredients: list[str] = Field(min_length=1)

    @field_validator("name")
    @classmethod
    def name_must_be_clean(cls, value):
        if "idiot" in value.lower():
            raise ValueError("name contains inappropriate language")
        return value

def check_recipe(data):
    try:
        recipe = Recipe.model_validate(data)
        return "valid: " + recipe.name
    except ValidationError as error:
        problems = []
        for problem in error.errors():
            field = ".".join(str(part) for part in problem["loc"])
            problems.append(field + " (" + problem["type"] + ")")
        return "invalid: " + ", ".join(problems)

tests = [
    {"name": "Pasta", "cooking_time_minutes": 10, "ingredients": ["pasta"]},
    {"name": "Pasta", "cooking_time_minutes": 0, "ingredients": ["pasta"]},
    {"name": "Pasta", "cooking_time_minutes": 10, "ingredients": []},
    {"name": "Idiot Pasta", "cooking_time_minutes": 10, "ingredients": ["pasta"]},
]

for data in tests:
    print(check_recipe(data))

Output

valid: Pasta
invalid: cooking_time_minutes (greater_than)
invalid: ingredients (too_short)
invalid: name (value_error)

Understanding the Code

• Field(min_length=1, max_length=50) says the name must have between 1 and 50 characters.
• Field(gt=0, le=480) says the cooking time must be greater than 0 and at most 480 minutes (8 hours).
• Field(min_length=1) on the list says there must be at least one ingredient.
• @field_validator(“name”) attaches our own function to the name field. The function raises a ValueError when the name contains a banned word. Pydantic turns that into a validation problem with the type value_error.
• check_recipe returns a short summary of what is wrong, built from the list of problems.
• Each test breaks a different rule, and the output shows which field failed and why.

Example 4: Retry Until the Reply Is Valid

In Chapter 7, you learned to retry when a reply fails a check. We can do the same with structured output. The fake model below first gives plain text, then a recipe with an impossible cooking time, and finally a correct recipe.

from pydantic import BaseModel, Field, ValidationError

class Recipe(BaseModel):
    name: str = Field(min_length=1, max_length=50)
    cooking_time_minutes: int = Field(gt=0, le=480)
    ingredients: list[str] = Field(min_length=1)

def make_fake_llm(replies):
    reply_iter = iter(replies)
    def fake_llm(prompt):
        return next(reply_iter)
    return fake_llm

def get_recipe(prompt, llm, max_attempts=3):
    for attempt in range(1, max_attempts + 1):
        reply = llm(prompt)
        try:
            recipe = Recipe.model_validate_json(reply)
            print("Attempt", attempt, "| valid")
            return recipe
        except ValidationError:
            print("Attempt", attempt, "| invalid")
    return None

llm = make_fake_llm([
    "Sure! Here is a recipe for pasta.",
    '{"name": "Pasta", "cooking_time_minutes": 0, "ingredients": ["pasta"]}',
    '{"name": "Pasta", "cooking_time_minutes": 10, "ingredients": ["pasta", "salt", "water"]}',
])

recipe = get_recipe("Give me a pasta recipe as JSON", llm)

if recipe is None:
    print("Sorry, I could not get a valid recipe.")
else:
    print(recipe.name, "-", recipe.cooking_time_minutes, "minutes")

Output

Attempt 1 | invalid
Attempt 2 | invalid
Attempt 3 | valid
Pasta - 10 minutes

Understanding the Code

• get_recipe asks the model for a reply and validates it. If validation fails, it asks again, up to max_attempts times.
• The first reply is plain text, so it fails. The second reply is JSON but has a cooking time of 0, so it fails the rule gt=0. The third reply passes.
• If all attempts fail, the function returns None, and the caller shows a safe message. This is the same fallback idea you used in Chapter 7.
• When the function succeeds, the program receives a real Recipe object whose values are guaranteed to follow the rules.

Tips for Real Applications

• Tell the model the exact format you expect in your instructions. Validation is the safety net, and clear instructions reduce how often the net is needed.
• Pydantic can generate a description of your model for you with Recipe.model_json_schema(). Many AI providers can use such a schema to make the model follow your structure more closely.
• When a retry is needed, you can send the validation problems back to the model and ask it to fix them.
• Validation checks the shape and the rules of the data, but it cannot tell whether the content is true. A well-formed recipe can still contain a wrong cooking time.

Key Takeaways

• Format guardrails make sure the AI’s reply has the structure your program expects.
• Pydantic models describe the expected fields and types, and model_validate_json checks JSON replies.
• Field rules and field validators let you add limits and custom checks.
• Catch ValidationError, then retry or fall back safely.
• Valid structure does not guarantee correct facts.

What is Next?

In the next chapter, you will learn how to detect and mask personal information, known as PII, such as emails, phone numbers, and card numbers, in both messages and replies.

If you liked the tutorial, spread the word and share the link and our website, Studyopedia, with others.

For Videos, Join Our YouTube Channel: Join Now

Read More:

• Generative AI Tutorial
• AI Ethics
• Machine Learning Tutorial
• Deep Learning Tutorial
• Ollama Tutorial
• Retrieval Augmented Generation (RAG) Tutorial
• ChatGPT Tutorial
• Microsoft Copilot Tutorial