Structured Output Validator

Project Overview

Structured Output Validator is a production-style AI engineering project that teaches how to safely consume LLM responses inside real software systems. LLMs naturally generate free-form text, but production applications often need strict JSON structures. This project shows how to request structured output, parse raw model responses, validate them using schemas, handle malformed responses, retry failed generations, and return reliable typed data to the frontend.

Problem Statement

LLMs are excellent at generating natural language, but real applications cannot depend on raw text responses. A backend service, fraud detector, resume matcher, SEO analyzer, product insight dashboard, or agent workflow usually needs predictable structured data. This project solves the problem of converting unstructured LLM completions into schema-safe JSON using validation models, retry policies, repair prompts, and frontend inspection tools.

"Yes, this looks like a scam because the message asks for urgent payment."

{
  "classification": "scam",
  "confidence": 0.87,
  "risk_factors": ["Urgency", "Payment request", "Unknown sender"],
  "safe_action": "Do not click the link"
}

What You Will Build

Concepts You Will Learn

Project Requirements

System Architecture

The frontend never talks directly to the LLM provider. It sends the user input and selected schema to the FastAPI backend. The backend builds the prompt, calls the LLM, parses the response, validates it using Pydantic, and returns either a valid typed JSON response or detailed validation errors.

Backend Design

apps/api
├── main.py
├── routes
│   └── validate.py
├── schemas
│   ├── scam_detection.py
│   ├── product_review.py
│   ├── resume_match.py
│   └── generic_extraction.py
├── services
│   ├── llm_client.py
│   ├── prompt_builder.py
│   ├── output_parser.py
│   ├── validator.py
│   └── repair_engine.py
├── core
│   ├── config.py
│   └── errors.py
└── tests
    ├── test_validator.py
    └── test_repair_engine.py

Frontend Design

apps/web
├── app
│   └── structured-output
│       └── page.tsx
├── components
│   ├── PromptInput.tsx
│   ├── SchemaSelector.tsx
│   ├── ValidationResult.tsx
│   ├── JsonViewer.tsx
│   ├── ErrorInspector.tsx
│   └── RetryTimeline.tsx
├── lib
│   └── api.ts
└── types
    └── structured-output.ts

API Contract

{
  "input": "Your account will be blocked. Click this link immediately to verify payment.",
  "schema_type": "scam_detection",
  "max_retries": 2
}

{
  "status": "valid",
  "schema_type": "scam_detection",
  "retry_count": 0,
  "data": {
    "classification": "scam",
    "confidence": 0.92,
    "risk_factors": [
      "Urgency",
      "Suspicious link",
      "Account threat"
    ],
    "safe_action": "Do not click the link"
  }
}

{
  "status": "invalid",
  "schema_type": "scam_detection",
  "retry_count": 2,
  "errors": [
    {
      "field": "confidence",
      "message": "Value must be between 0 and 1"
    }
  ],
  "raw_output": "{ \"classification\": \"scam\", \"confidence\": 120 }"
}

Example Pydantic Model

from typing import Literal
from pydantic import BaseModel, Field

class ScamDetectionOutput(BaseModel):
    classification: Literal["scam", "safe", "suspicious"]
    confidence: float = Field(ge=0, le=1)
    risk_factors: list[str]
    safe_action: str

Validation Lifecycle

Common Failure Cases

Interview Explanation

After Building This Project, You Can Explain

• Why raw LLM output is risky in software systems
• Why structured output is needed and how to design it
• How schema validation works at the API gateway layer
• How Pydantic validates LLM responses in Python
• How retry and repair logic improves AI reliability
• How frontend interfaces inspect validation success/failure
• How to explain this project during technical AI interviews

Build Milestones

Future Improvements

• •Add multiple LLM provider support (Anthropic, Gemini, local models)
• •Add streaming output validation handling chunked tokens in flight
• •Add schema generation from natural language input descriptions
• •Add validation analytics dashboard monitoring latency and accuracy
• •Add saved validation history tracking logs
• •Add user-defined custom schemas sandbox
• •Add LangChain / Instructor / Guardrails comparison reference documentation
• •Add agent tool-call validation support mapping parameters
• •Add observability telemetry tracking validation failure rates
• •Add cost tracking for retry attempts