Case study — Verdict
Air-cargo compliance pre-check for a Sydney freight forwarder. Solo build, April 2026 – present.
The problem
Freight operators triage shipment paperwork by hand: airway bills, invoices, packing lists, dangerous-goods declarations — checked against biosecurity windows, prohibited-imports rules and air-transport restrictions, under deadline. Misses are expensive: held cargo, re-lodgement, penalties. The people doing this work don’t need an oracle; they need a tireless pre-checker that shows its working.
The constraint that shaped everything
A compliance verdict you can’t reproduce is a liability. So the system is built on one rule: the model never decides. The LLM extracts fields, classifies documents and drafts follow-ups. Every pass / warning / blocked verdict comes from a deterministic rule engine — pure functions over validated fields, versioned with SHA-256 snapshots, so any verdict can be replayed bit-for-bit later.
Architecture
What decides what
| Concern | Deterministic rules | LLM | Human |
|---|---|---|---|
| Compliance verdicts (pass/warn/block) | ● | — | review queue |
| Field extraction from documents | schema validation | ● | corrections UI |
| Document classification | keyword heuristic first | fallback on low confidence | — |
| Follow-up emails | template engine | drafting assist | always reviewed |
| Risk scoring | ● | — | — |
Built & measured
In plain terms: Verdict reads the shipping paperwork, checks it against the real biosecurity, dangerous-goods and customs rulebooks, and shows its working — with a copyable receipt — on every call. The specifics, for engineers:
9 document types · 57 rules across 3 modules, encoding ISPM-15, DAFF BMSB seasonal measures, IATA DGR (65th ed.) with a ~190-entry UN air whitelist, Prohibited Imports Regulations, WCO HS and Incoterms 2020 · 15 authenticated REST endpoints (JWT, RBAC, IDOR protection, audit logging) · bilingual PDF + OCR ingestion (English/Chinese) · table-driven test suite (~1,000 cases incl. 8 adversarial rounds) gated by CI on test, type-check, build and lint · extraction measured at 96.3% strict / 99.0% semantic on the 511-field corpus above.
Data boundary & provider choice
The LLM layer is a hand-rolled, OpenAI-compatible client behind a provider abstraction — currently DeepSeek (chosen on cost during development; a full 85-document eval run costs cents) with automatic failover to MiniMax. Synthetic and anonymised data only during development; no client PII in prompts. Because the abstraction is the contract, swapping to a sovereignty-compliant provider — Azure OpenAI (Sydney), AWS Bedrock, or a local model — is a configuration change, not a rewrite. The deterministic core narrows the blast radius either way: whatever the provider, it only ever drafts and extracts. It never decides.
Honest limitations
Mock-data demo above; the real system runs privately for one client and is not a public product. Accuracy is measured on clean digital English PDFs from that client’s document mix — not scans or photos. Rules encode my reading of public regulations, verified by tests, not by a licensed broker. No production-traffic claims.
Roadmap (designed, not built)
- designed HS-code classification with tariff retrieval (pgvector on the existing Postgres)
- designed compliance copilot for explain-why queries (read-only)
- designed email-thread intake orchestration
Listed as design intentions — nothing here is implemented, and saying so is the point.
Status verified 2026-06-12