Yeda AI Tips · #128

Regex First, Model for the Edges

Do not send every line of structured text to a language model. Regex handles most of it at a fraction of the cost. For consistent-format text — forms, quizzes, invoices, logs — a well-tuned set of patterns resolves roughly 95–98% of cases deterministically, and teams commonly report order-of-magnitude (~20×) lower per-record cost than an LLM-only pipeline. The model earns its keep on the 2–5% that regex can't handle.

Why LLM-on-everything is the wrong default

An LLM call for a field you could match with \d{2}/\d{2}/\d{4} buys you three downsides at once:

Cost — you pay per token on every record, even the trivially regular ones.
Latency — a compiled regex matches in microseconds; a model round-trip is hundreds of milliseconds or more.
Nondeterminism — the same invoice can parse two different ways on two runs. Regex gives you the same answer every time, which is exactly what you want for the 95%+ of records that follow the format.

The hybrid pipeline

Parse with regex first. Score each extraction for confidence. Send only the low-confidence ones to the model. Everything else ships straight from the regex.

import re

AMOUNT = re.compile(r"(?i)total[:\s]*\$?(\d{1,3}(?:,\d{3})*(?:\.\d{2})?)")

def extract(line: str) -> dict:
    m = AMOUNT.search(line)
    if m:
        return {"amount": m.group(1), "confidence": 1.0, "source": "regex"}
    # No match, ambiguous match, or failed validation -> queue for the LLM
    return {"amount": None, "confidence": 0.0, "source": "llm_queue"}

results = [extract(line) for line in lines]
edge_cases = [r for r in results if r["confidence"] < 0.8]
# Only edge_cases go to the model — typically ~2–5% of records.

For the LLM leg, ask for a JSON-schema-constrained response (structured outputs) so the fixed edge cases merge back into the same shape as your regex output.

Rules of thumb

Situation	Route
Format repeats consistently (forms, invoices, logs)	Regex, always
Field matched and passes validation (checksum, range, date parses)	Ship it — no model call
No match, multiple conflicting matches, or validation fails	Send to the LLM
Free-form prose, no repeating structure	LLM from the start — regex isn't the tool
Latency-tolerant edge cases	Batch them: OpenAI's Batch API is 50% cheaper, ≤24h turnaround

Power-user moves

Score confidence with cheap signals. "Matched + validated" (date parses, checksum passes, value in expected range) is high confidence. "Matched but out of range" or "two patterns disagree" is low — route it to the model.
Batch the edge cases. If the low-confidence queue doesn't need real-time answers, submit it through a batch endpoint at a 50% discount with a 24-hour window.
Let the misses tighten the regex. Log every record that fell through to the LLM. Recurring miss shapes become new patterns, and your regex coverage climbs over time — the model bill shrinks month over month.
Test patterns like code. Keep a fixture file of real matched and missed lines and run it in CI. A regex edit that silently drops coverage is the same class of bug as a broken function.
Anchor and validate. re.fullmatch (or ^...$) beats a loose search for whole-field extraction; a match that then fails a semantic check (impossible date, negative total) should still route to the model.

Resources

Building an AI feature? Yeda AI designs, audits, and ships production LLM systems.

Talk to us · Read the blog