Andrew Xia · https://hh0hh.com
Case study — Verdict
Verdict is an independent prototype for reviewing shipment paperwork. I designed the workflow, system boundaries and evaluation, using AI-assisted implementation. It brings conflicting document fields and their supporting evidence into one review flow.
The problem
Shipment information arrives across airway bills, invoices, packing lists and declarations. A reviewer has to spot missing documents and conflicting quantities, weights or parties, then prepare a clear handover. I built a workflow that brings those comparisons together and keeps the supporting details visible.
Designed around the review, not just the model
Access and change boundaries
Authentication is followed by case-owner checks. Completed pre-checks must be reopened before reviewed inputs can change.
Failures stay local to a file
Bounded retries and per-file failure handling keep one extraction failure from discarding the whole batch.
Findings lead back to their basis
Findings identify the rule, documents, fields and values behind a conflict. A person verifies them and reviews follow-up drafts before acting.
My central design decision
Keep extraction and judgment separate: the model proposes fields; written rules issue the system verdict. Schema validation checks structure, not whether a field is true. Compliance-review reports identify that workflow’s rule-set snapshot with SHA-256; this identifies rule bytes, not correctness.
Architecture
-
Input
Documents
Bring documents together while retaining field sources.
-
AI periphery
Propose fields
The model extracts fields. Extracted values can still be wrong.
-
deterministic core
Check structure
Validate shape and types, not whether the content is true.
-
deterministic core
Apply written checks
Written checks assign system status and identify findings.
-
Human judgment
Human review
A person checks evidence and gaps, then decides what happens next.
Assisted drafting · outside the deterministic decision core
Templates and model assistance prepare follow-ups for review before sending or acting.
Traceable records
- audit log
- field corrections
- document versions
Records make inputs and changes inspectable; they do not establish truth.
Explore implementation details
- Document classification
- heuristic classifier → LLM fallback
- Model extraction
- JSON-mode extraction
- Structure validation
- Zod schema validation
- deterministic checks
- (57 across two workflows)
- Rule output
- review status + findings
What decides what
| Concern | Deterministic rules | LLM | Human |
|---|---|---|---|
| System review status (pass/review/block) | ● | — | verify findings; decide next steps |
| Field extraction from documents | schema validation | ● | corrections UI |
| Document classification | keyword heuristic first | fallback on low confidence | — |
| Follow-up emails | template engine | drafting assist | always reviewed |
| Rule-based score (not a probability) | ● | — | interpret with findings |
Recorded verification
The prototype documents 9 document types and 57 checks across two workflows. Project records list ~1,000 test cases and 8 adversarial rounds, with CI for tests, types, build and lint—not complete real-world coverage. The 2026-06 evaluation measured 96.3% field agreement (normalisation and numeric tolerance) on 511 fields of synthetic English text; 99.0% also allowed substring containment. This does not measure PDF parsing or OCR.
Data boundary & provider choice
Model calls sit behind a provider interface, separate from the decision engine. DeepSeek and MiniMax are implemented and selected by configuration, with bounded retries within a provider—not automatic cross-provider failover. A new backend requires an adapter and validation. The public examples use synthetic data; any operational deployment would need a separate review of data handling, hosting and provider requirements.
Evaluation scope & project status
Independent prototype, in development. The public examples and published evaluation use synthetic fixtures. The recorded evaluation covers clean English text reconstructed from hand-authored ground truth, not PDF parsing, OCR, scans, photos, other layouts or real operational documents. System checks support review; they do not authorise shipment or replace professional judgment. The rule implementation is not professional or regulatory validation. No production-traffic performance is claimed.
Roadmap snapshot · 2026-06-12 (building · designed)
- building HS-code classification: deterministic layer running (keyword matcher over a 5,613-code AHECC dataset, plus tariff and FTA lookup, behind three endpoints). The AI-assisted path above it is a stub that returns nothing, and there is no UI
- designed compliance copilot for explain-why queries (read-only)
- designed email-thread intake orchestration
Historical status record: 2026-06-12