FreshFlow Analytics

Conversational analytics for Meridian Markets' shrink, sales, and shipment data — built so that every answer is traceable to a real query, every business assumption is stated rather than silently baked in, and every response carries a confidence signal a merchandiser can actually act on.

This is a portfolio prototype using synthetic retail data. It is designed to show the architecture of a trustworthy natural-language analytics system rather than to represent any real retailer, dataset, or production deployment.

The premise: an LLM should never be trusted to know what "shrink" means, what counts as "the top departments," or how a rate is denominated. It should be trusted to translate a question into a query against a semantic layer that already encodes those decisions — the same way an analyst looks up a metrics dictionary instead of re-deriving a formula from memory each time.

What it does

Ask it things like:

• "What were our top 10 items by shrink last month?"
• "How did strawberry sales trend over the three months?"
• "Which department has the highest shrink rate?"
• "Why is dairy shrink up in the Northeast in June?"

Follow-up questions inherit context automatically — "now just Northeast," "what about by store instead" — via an explicit, inspectable session-state object, not a re-read transcript. Every answer arrives with the SQL that ran, the actual rows it returned, the business assumptions applied, and a deterministic confidence label.

Architecture

flowchart TD
    A[CSV extract] --> B["Data Layer (DuckDB)"]
    B --> C["shrink_facts view<br/>(built from business_rules.yaml)"]
    C --> D[Context Assembly]
    D --> E["NL → SQL Generation<br/>(Claude, tool use)"]
    E -- "resolve_entity tool" --> F[Entity Resolution]
    F --> E
    E -- "emit_query_plan tool" --> G[Execution]
    G --> H["Answer Formatting<br/>(Claude)"]
    H --> I["Groundedness Check<br/>(deterministic)"]
    I --> J[Response Assembly]
    J --> K[Session State Update]
    K -.->|feeds back into next turn| D

    style C fill:#1f4e3d,color:#fff
    style I fill:#4a2c0f,color:#fff

Loading

The semantic layer — where trust actually comes from

The model's SQL-writing surface is deliberately narrow. Meridian's business logic — how shrink is defined, which departments count as "fresh," what a shrink rate is measured against — lives in exactly one place: config/business_rules.yaml. That file is read by two independent consumers that cannot drift apart because they share one source:

1. data/load.py renders it into data/views.sql, which builds the shrink_facts DuckDB view — a single, hand-verified join across items, shipments, and sales_daily that computes both cost and unit shrink, flags default department scope, and correctly handles days with no movement (a FULL OUTER JOIN on store-item-day, not a naive cross join).
2. pipeline/context.py embeds the same YAML verbatim — not paraphrased — into the prompt that drives SQL generation, so the model is reading the identical text a reviewer could diff against the view definition.

The model's job shrinks to "write a filter and group-by over a pre-verified view," which is both more reliable and trivially explainable: the join logic was written once, tested once, and the model only ever writes the easy part.

Pipeline

Stage	Module	Responsibility
1. Data layer	data/load.py	Loads the raw extract into DuckDB, generates shrink_facts from business rules
2. Context assembly	pipeline/context.py	Schema + business rules + live session state, concatenated into one prompt
3. NL → SQL generation	pipeline/generate_sql.py	Claude, driven via forced tool use — never free-text SQL, always structured output
4. Entity resolution	pipeline/entity_resolution.py	Fuzzy-matches loose product references against the catalog; surfaces ambiguity instead of guessing
5. Execution	pipeline/execute.py	Statement-whitelisted, row-capped, timeout-bounded DuckDB execution
6. Groundedness check	pipeline/groundedness.py	Deterministic numeric verification of the generated answer — no second LLM call
7. Answer formatting	pipeline/format_answer.py	Turns a result set into plain language, then assembles the final response object
8. Session state	pipeline/session_state.py	Explicit structured memory, updated by a diff the model returns each turn

NL → SQL generation as a structured tool-calling conversation

Claude is never asked to just "write SQL" and hope the response parses. It's given two tool definitions and forced to call one on every single turn (tool_choice: {"type": "any"}) — free-text replies aren't a possible output at all at this stage:

• resolve_entity — callable mid-conversation whenever a question names a product in loose language. Returns a single match, multiple candidates, or no match. On ambiguity (e.g. "strawberry" matching both Strawberries 1lb and Strawberry Yogurt), the system asks for clarification rather than guessing — a confidently wrong item match is a worse failure than a one-line question back to the user.
• emit_query_plan — the terminal call. Its structured input is the pipeline's output: the SQL to run (one entry for a simple aggregation, several labeled entries for a diagnostic question that decomposes into sub-queries), the assumptions applied, a self-reported question type, and a session-state diff expressed as three explicit sets — fields carried forward, fields changed, fields reset — with a stated reason for every reset.

The tool schemas themselves carry real structure, not just flat strings: needs_clarification is a genuinely nullable field ("type": ["string", "null"]), and the sql field is an array of {label, sql} objects so a diagnostic question can decompose into several independently labeled, independently executed sub-queries rather than forcing a "why" question through a single SELECT.

Because a tool's result has to be handed back to the model in a subsequent request — the API has no way to call a function and see its return value in the same round trip — this runs as a short, bounded conversational loop: Claude calls resolve_entity, the result is appended to the message history as a tool result, and Claude is invoked again with that fuller context, repeating until it emits a final query plan or a hard iteration cap is hit.

Groundedness — verification without another model call

This is the actual trust mechanism, and it's deliberately boring: no LLM judges the LLM. Instead:

1. Every dollar figure, percentage, and count is regex-extracted from the generated answer, with its exact character span.
2. Each is checked against the real numeric cells the executed query returned — plus every pairwise difference between two values in the same result column, so a completely ordinary statement like "shrink is down $1,025 month over month" grounds correctly even though no single row states that figure; it's the plain arithmetic difference between two rows the query already returned.
3. Labeled Grounded when every number matches, Ungrounded when a stated figure doesn't appear anywhere in the results, or Partially grounded for diagnostic questions — where the individual figures check out but the causal narrative connecting them ("driven by X") is a synthesis across several sub-queries, not a single directly-queried fact.

An ungrounded answer is never withheld — per the internal-tool trust model, the confidence label is always shown and the user is trusted to weigh it, the way they'd weigh a colleague's hedge. The UI highlights the exact unmatched substring in the answer and, where a plausible one exists, shows the closest real value from the results — never a numerically-nearest-but-irrelevant figure standing in for a real match.

Execution guardrails

Generated SQL never runs unexamined. Before anything reaches the database:

• Statement whitelisting — only SELECT/WITH statements are permitted at all; any mutating keyword (INSERT, UPDATE, DELETE, DROP, ALTER, CREATE, and a dozen others) is rejected outright, regardless of where in the query it appears.
• Row capping — a query with no LIMIT is transparently wrapped (SELECT * FROM (<generated query>) LIMIT 1000) so a broad aggregation can't return an unbounded result set.
• Wall-clock timeout — execution runs under a bounded wait; a runaway query can't hang a request indefinitely.
• Self-correcting retry — if the generated SQL fails to execute (a mistyped column, for instance), the exact error is fed back to the model as additional context and it gets one corrective attempt before the pipeline fails gracefully with a clear message — the same recovery path a human analyst would take after a typo, rather than surfacing a raw database exception to the end user.

Session state — an explicit object, not a transcript

Multi-turn memory is a structured object (measure, scope_depts, rate_denominator, date_period, resolved_entities, filters), and every turn Claude returns a diff against it — carried_forward, changed, reset — rather than the system re-inferring intent from raw conversation history. That distinction matters: a sliding-window transcript has a well-known failure mode where a filter from several turns back silently bleeds into a question it no longer applies to. Here, that decision is explicit and visible in the UI on every turn, so a wrong carry-forward is a visible, attributable model decision — not a hidden architecture flaw.

Operational resilience

Every call to the language model, and the database execution step, is wrapped in layered protection rather than hoping the happy path always holds:

• Retry with backoff — the model client's own built-in retry handles transient failures (rate limits, timeouts, 5xx responses) at the API layer. A second, independent retry policy covers the database execution step specifically, with a predicate that distinguishes transient failures (worth a couple of quick attempts) from permanent ones — a malformed query will never succeed on a second try no matter how many times it's retried, so those fail immediately instead of wasting time and attempts on a foregone conclusion.
• Circuit breaking — both model call sites are wrapped by a circuit breaker: after five consecutive failures it trips open and fails fast for a cooldown window, rather than letting every subsequent request queue up behind a dependency that's already down. Verified by deliberately forcing failures and confirming the breaker actually opens, not just configured and assumed to work.
• Rate limiting — the chat endpoint is capped per client, since this is a shared internal tool where one runaway script shouldn't be able to exhaust the team's model quota for everyone else.
• Idempotency — every request carries an ID; a repeated ID within a short window returns the cached response instead of re-running the full pipeline (and re-billing the model calls) a second time for what was actually a network retry, not a new question.
• Graceful degradation over hard failure — at every stage where something external can go wrong (a bad query, an unreachable model, ambiguous input), the system's default is a clear, honest message back to the user rather than an unhandled exception.

Interface trust affordances

The chat interface is a deliberately plain single page — no framework, no build step — because the actual product decisions live in what it exposes, not how it's styled:

• Every assistant message carries a confidence badge and a collapsible detail panel showing the exact SQL that ran, the real result rows it returned (not a paraphrase), the business assumptions applied, and the session-state diff for that turn.
• When an answer is flagged less than fully grounded, the specific unsupported substring is highlighted directly inline in the answer text — not just a generic warning banner — with a tooltip showing the closest real value found in the data, when one exists and is plausible enough to be worth showing. A number that's off by an order of magnitude gets flagged as unsupported rather than paired with a misleading "closest" figure that isn't actually close.

Project layout

freshflow/
  data/
    load.py            # CSV -> DuckDB tables, business_rules.yaml -> shrink_facts view
    views.sql           # generated view definition (checked in, inspectable)
  config/
    business_rules.yaml # the single source of truth for shrink definitions
  pipeline/
    context.py           # [3] context assembly
    generate_sql.py       # [4] NL -> SQL, tool-use conversation
    entity_resolution.py  # fuzzy product-name matching
    execute.py             # [5] guarded execution
    groundedness.py         # [6] deterministic verification
    format_answer.py         # [7] answer text + response assembly
    session_state.py          # [8] explicit state + diff application
  infra/
    db.py                # shared DuckDB connection + parsed rules, built once
    resilience.py          # LLM client, circuit breaker, cache, rate limiter
  static/
    index.html            # chat UI — vanilla HTML/CSS/JS, no build step
  app.py                   # FastAPI entrypoint

Running it

pip install -r requirements.txt

Set an Anthropic API key in .env. Do not commit real keys:

ANTHROPIC_API_KEY=your-anthropic-api-key

Start the server:

python -m uvicorn app:app --reload

Open http://localhost:8000 for the chat interface, or drive the pipeline directly:

import app
response = app.answer_question("What were our top 10 items by shrink last month?")

python run_eval.py runs the full four-sequence, twelve-turn regression set end to end and prints the SQL, assumptions, confidence label, and state diff for every turn — useful for eyeballing behavior directly.

For automated testing, pytest runs the deterministic suite (groundedness, session_state, execute's guardrails, entity_resolution) with no API key and no network in a few seconds. The same four eval sequences also exist as pytest-native tests in tests/test_eval_sequences.py, tagged @pytest.mark.llm and run with pytest -m llm — they call the live model, so they cost money and use structural assertions rather than exact wording, since model output varies run to run.