A durable, citation-backed multi-agent research system built on LangGraph and shipped as a small FastAPI service. Ask it a question; it plans subtasks, fans out to parallel web-research workers, critiques its own coverage and loops on the gaps, then synthesizes a report where every claim is tied to a real source — all behind layered input/output guardrails, a hard budget ceiling, and an optional human-in-the-loop approval gate.

It is deliberately built as a production-shaped portfolio piece: observable (trace tree per run), measurable (LLM-as-judge evals gated in CI), trustworthy (guardrails + retries), and durable (checkpointed runs that survive restarts and can pause for a human).

What it does

Given a research question, the system runs a stateful agent graph rather than a linear prompt chain:

1. Input guard — a cheap classifier rejects out-of-scope or unsafe queries before any money is spent.
2. Planner — decomposes the query into 3–5 focused, non-overlapping subtasks using structured output.
3. Approval gate (optional) — pauses for a human to approve / edit / reject the plan before the expensive fan-out. Off by default.
4. Parallel researchers — one ReAct sub-agent per subtask, each with a web search tool, all running concurrently.
5. Critic — assesses whether the aggregated findings actually cover the query, and loops back to the planner with notes if there are gaps.
6. Synthesizer — produces a structured report (summary + sections + citations) grounded in the gathered sources.
7. Output guard — verifies citations are real and scans for leaked prompts/PII; sends the report back for repair if it fails, then ships with flags if needed.

The whole run is checkpointed after every step, so it survives process death, can be resumed, and can pause indefinitely at the human gate.

Capabilities at a glance

Capability	Where it lives
Agentic orchestration — parallel fan-out, bounded reflection loop, nested sub-graphs	graph.py
Multi-provider / multi-model — a different model per role, swappable by env var	models.py, config.py
RAG-style grounding & attribution — tool-using researchers + cited synthesis	nodes/researcher.py, nodes/synthesizer.py
Layered guardrails & governance — input scope/safety, output citation/PII, budget ceiling, retries	guardrails.py, nodes/guardrails.py
LLMOps: evaluation + CI gate — LLM-as-judge metrics fail the PR on regression	evals/judges.py, .github/workflows/evals.yml
Observability — full trace tree per run via LangSmith / OpenTelemetry	env-driven
Durability + human-in-the-loop — checkpointer, pause/resume, approval gate	persistence.py, nodes/approval.py
Production deployment — FastAPI start-then-poll API, reproducible Docker image	api.py, Dockerfile

Architecture

The system is modelled as a state machine, not a pipeline. A single typed state object (ResearchState) travels through the graph; each node reads it and returns a partial update merged back in. This is what lets the graph express what a linear chain cannot: cycles (the critic's reflection loop), branches (guard and approval decisions), and parallelism (the researcher fan-out).

Two state-design choices make the rest possible:

• Reducers on concurrently-written fields. findings is an Annotated[list[Finding], add] and llm_calls is Annotated[int, add]. When N researchers write in the same superstep, the reducer accumulates their writes instead of letting the last one silently overwrite the rest.
• Transient routing flags (_sufficient, _plan_rejected, _aborted) keep control-flow signals out of the durable contract while still driving conditional edges.

Each researcher node is itself a small agent that reasons and acts in a loop: decide what's needed, call the web-search tool, read the result, repeat until it can answer — built with LangChain's prebuilt create_agent, with a callback that counts internal model calls so the budget guard sees each branch's true cost.

A run takes tens of seconds, so the API is start-then-poll: POST returns a thread_id immediately and the graph runs in a background thread, writing checkpoints as it goes. Because the result is read back from the checkpointer, a GET returns the report even after a server restart.

Tech stack

Layer	Choice	Why
Orchestration	LangGraph	Explicit state-machine graphs — cycles, branches, and dynamic parallelism cleanly.
LLM abstraction	LangChain init_chat_model	One provider-agnostic call; switch providers by changing a string.
Models	OpenAI (default), swappable to Groq / Gemini / Ollama	Per-role selection: cheap model for researchers, stronger model for planner/synth.
Web search	Tavily	Clean, LLM-ready text instead of raw HTML.
API	FastAPI + Uvicorn	Async, auto OpenAPI docs, clean 202 + poll semantics for long runs.
Persistence	LangGraph SqliteSaver (dev) / PostgresSaver (prod)	Durable checkpoints unlock resume, crash recovery, HITL.
Validation & config	Pydantic + pydantic-settings	Typed state, structured LLM outputs, env-driven config.
Evaluation	pytest + LLM-as-judge	Framework-free quality gate that runs anywhere.
Observability	LangSmith / OpenTelemetry	Trace tree per run; compare providers on identical code.
Packaging	uv + Docker (python:3.12-slim)	Lockfile-pinned, reproducible, fast builds.
CI	GitHub Actions	Runs the eval harness on every PR, fails on quality regression.

Design decisions & the why

A provider-agnostic, model-per-role factory. Every node calls get_model("planner"), never a provider class directly — a registry maps logical roles to provider:model specs. Researchers run many calls under fan-out and use a cheap/fast model; the planner and synthesizer use a stronger one. Switching providers is an env-var change with zero code changes.

State-first design with reducers. The state contract was designed before the nodes, and findings got an add reducer from day one — before parallelism existed. Without it, N researchers writing in the same superstep would silently overwrite each other's data.

Dynamic fan-out with Send. The planner decides at runtime how many researchers to spawn; the graph emits one Send per subtask. This is the feature that separates "chained some prompts" from "built an agentic system."

A bounded reflection loop. The critic can send the graph back to the planner to fill gaps — a cycle bounded by an iteration counter and a hard LLM-call budget (max_research_iterations, max_llm_calls_per_run in config), so a report always ships.

Traceable citations via structured output. Findings carry their source_url through the pipeline, and the synthesizer emits structured output (summary / sections / citations) so each report is grounded and attributable.

View on GitHub