IThe case against generated SQL

The model shouldn't
write the SQL.

Text-to-SQL is the obvious way to point an agent at a database, and the demo is genuinely impressive. Production is where it turns: the same question compiles to different SQL, a quiet join error moves revenue 15%, and the model's output becomes an attack surface. There is another path.

IICredit first

Text-to-SQL earned the hype. In its lane.

Where it's the right tool

  • Prototypes on throwaway data
  • Exploring an unfamiliar schema, human watching
  • One-off questions someone will sanity-check
  • The demo — it really does demo well

Where it breaks

  • Numbers someone will act on
  • Untrusted text anywhere in the context
  • Production credentials on the connection
  • Answers that must be reproducible or audited
  • Agents that run unsupervised

The pattern isn't wrong. The blast radius is. Every hazard on this page traces to one design choice: the model's output is executed as code.

IIIThe two paths

Same question. Two architectures.

Follow one question down both paths — a free-form SQL string on the left, a typed plan on the right. Numbered stamps point into the hazard ledger below.

the question

“Which region has the highest total revenue?”

Path A

The model writes SQL

  1. the model improvises a string 010304

    Whatever the model read — a user message, a retrieved row — can shape this string.

    run 1SELECT region, SUM(total) FROM orders GROUP BY region;run 2 · same questionSELECT o.region, SUM(i.amount) FROM orders o LEFT JOIN order_items i ON i.order_id = o.id GROUP BY 1;
  2. your connection executes it 05

    With the connection's full authority. Read, join — and unless someone remembered the flag, write.

the answer

run 1 → east · 2130.50

run 2 → east · 2450.08+15% — join fan-out 02

Two runs, two numbers. Both plausible. No signal which — if either — is right.

Path B

The model files a typed plan

  1. typed intent

    Not a string — a value with a schema. It can only name operations the contract defines.

    { "kind": "query", "version": "1", "source": "revenue", "op": "sum", "group_by": "region" }
  2. contract check

    Hash-pinned. An unknown operation returns unsupported_operation with nearest matches — never a guess.

  3. policy check

    Your code-level allow-list. A request can narrow the surface, never widen it.

  4. deterministic execution

    float64, single thread, pinned runtime. Same plan, same bytes, same answer.

the answer

east · 2130.50

plan_hash f87610d8afeb…

decision_path "exact_spec"

One answer, carrying the hashes to replay it — next week, next quarter, in either language.

IVThe hazard ledger

Five ways generated SQL goes wrong

  1. 01

    P2SQLarXiv 2308.01990

    Injection moved to the output channel

    Input sanitization inspects what goes into the model. P2SQL attacks arrive in what comes out: well-formed, malicious SQL assembled from instructions hidden in a user message or a retrieved row. No input filter ever sees it — the attack is the output.

  2. 02

    −15%the dashboard that lied

    Wrong joins fail silently

    A fanned-out join double-counts rows, and revenue reads 15% off. No exception, no warning — wrong SQL doesn't crash, it reports. The result is always a number, and a plausible number carries no signal that it's wrong.

  3. 03

    1 → none question, n queries

    Same question, different SQL

    Ask twice and the model may compile the question two different ways — sometimes to two different answers. There is no canonical query to review, cache, or replay. Yesterday's number can't be reproduced, even just to check it.

  4. 04

    91.2 → 21.3% correct · benchmark → enterprise schema

    The accuracy cliff

    On clean benchmark schemas, a frontier model wrote correct SQL 91.2% of the time. On real enterprise schemas: 21.3%. In the same research wave, roughly 40% of text-to-SQL agent runs failed outright or returned incorrect results. Benchmarks are tidy. Your schema isn't.

  5. 05

    1 prod DBdeleted by an agent — the Replit incident

    The write path was always there

    Replit's coding agent deleted a production database despite explicit instructions not to touch it. That's the lesson: a read-only instruction is a request. If the connection can write, the write path exists, and eventually one bad completion finds it. Read-only has to be a property of the tool, not a line in the prompt.

Don't filter the output.
Don't generate it.

No SQL string, no injection · no guess, no drift

VThe certificate

Five dimensions, side by side

Not an LLM-to-SQL generatorengine README

Certificate of difference

authoring surfacegenerated SQLa free-form SQL stringtyped plana typed, versioned intent
execution surfacegenerated SQLanything the connection allowstyped plan4,574 read-only capabilities
write pathgenerated SQLpresent unless blockedtyped plannone, by construction
governancegenerated SQLprompt-level, best-efforttyped plancode-level allow-lists the model can't widen
reproducibilitygenerated SQLnonetyped plana replay hash on every result

right column pinned to contractsha256:79f1c5a6…924be9a1

The fine print holds up: the engine's one escape hatch — getUnsafeRuntime — is absent from every model-facing surface, so a model can't reach it. And the README states the doctrine plainly: this is not an LLM-to-SQL generator.

VIQuestions

Asked in production

How do I block DELETE and DROP in LLM-generated SQL?

Don't filter the SQL — stop generating it. Denylists inspect strings, and models are endlessly inventive at producing new ones. SQAI removes the class instead: the model files a typed plan against 4,574 read-only capabilities, and no write capability exists for any plan to name.

What is P2SQL injection?

Prompt-to-SQL injection: malicious SQL that appears in the model's output, assembled from instructions hidden in whatever the model read — a user message, a document, a retrieved row. Input sanitization never sees it, because the attack lives in the output channel. It's documented in arXiv 2308.01990.

Is text-to-SQL ever the right choice?

Yes — for prototypes and supervised exploration on non-production data it's fast and genuinely useful. It's the wrong choice once the number will be acted on, the context contains untrusted text, or the answer has to be reproducible.

How do I give an AI agent read-only access to a database?

Make read-only structural, not configured. A read-only flag on a connection is a setting someone can change. SQAI's execution surface contains only read capabilities — the write path is absent by construction, and even the engine's escape hatch is unreachable from any model-facing tool.

How is a typed plan different from generated SQL?

A SQL string can say anything the grammar allows. A typed plan can only say what the contract defines. Every plan is validated against a hash-pinned contract, checked against your allow-list policy, and executed on a deterministic engine — with a replay hash on every result.

Take the other path.

No account. No key. Local data stays local.

Read the docs →