Invoices, inbox to accounting system
AI-Assisted Accounts Payable
Multi-site retail operator, Australia
A multi-site retail operator in Australia was receiving hundreds of supplier invoices, credit notes and statements every week into a shared Gmail inbox: PDFs, scanned cash receipts, invoices typed into the email body, and "click here to download" links. Their accounts team keyed each one into their accounting system by hand, matched it to the right site, and chased invoices that appeared on statements but never arrived. In three weeks we had a secure, rules-first invoice platform in production. It scans the inbox on a schedule, finds every accounting document (including ones inside links, combined PDFs and photographed receipts), uses AI to classify each document and extract the key fields with evidence quoted from the source, checks them against deterministic rules, and sends them to the accounting system as draft bills, with an optional human review step. It also reconciles supplier statements line by line, so missing invoices surface on their own instead of at month-end.
At a glance
- Client
- Multi-site retail operator (fuel and convenience), Australia
- Problem
- Manual entry of a high volume of supplier invoices from a shared inbox into the accounting system
- Solution
- Email-to-accounting invoice platform: AI classification and extraction, rule checks, optional human review and statement reconciliation
- Timeline
- 3 weeks from first commit to production, then weekly iteration with the accounts team
- Engineering
- 150+ shipped iterations, 46 database migrations, 600+ automated tests
- Stack
- Next.js 16TypeScriptPostgreSQLpg-bossGmail APIXero / QuickBooks APIAzure AI FoundryKamalAzure
The challenge
The client runs several retail sites, and each one buys from dozens of suppliers: fuel, groceries, beverages, newspapers, utilities, maintenance and services. Almost all of them send their paperwork to one shared accounts inbox, and no two do it the same way.
Format chaos
Clean PDFs, multi-invoice PDFs, photographed cash receipts with handwritten lines, invoices written in the email body, and emails holding nothing but a download link.
Mixed document types
Invoices arrive alongside credit notes, statements, remittances, order confirmations, direct-debit notices and marketing. Only some belong in accounts payable.
Right site, every time
One supplier can deliver to several sites, so each invoice must reach the correct site and accounting entity, usually from a "Ship to" block that varies by supplier.
Exclusions
Some suppliers and bill types are paid another way and must never be entered as bills, but they still land in the inbox.
Statements and missing invoices
Monthly statements list invoices the team may never have received. Finding those gaps meant cross-checking each line by hand.
Duplicates and credits
The same invoice often arrives twice, and credit notes sometimes look like invoices with a negative total.
Our approach
We treated this as a rules-first system rather than "AI that posts bills". The AI does the reading, deterministic code does the judging, and people stay in control: review is optional, and the team decides how much to check. We built it closely with the accounts team. Real supplier examples drove each iteration, and every edge case they flagged became a rule and a regression test.
What we built
Reliable mailbox intake
- Connects to the shared Gmail inbox with OAuth on the narrowest workable scope; refresh tokens are encrypted at rest.
- Per-site scan schedules with incremental watermarks, so a delayed or failed scan catches up and never skips mail. Leases and checkpoints recover interrupted runs without double-processing.
- Pause and resume, date-range rescans, and a run history that explains every email: processed, skipped (and why) or failed.
- Handled emails get a Gmail label, so the inbox shows at a glance what has been done.
Document discovery
- Inventories every nested attachment, inline image, email body and invoice link before analysis.
- Follows supplier download links safely: HTTPS only, every redirect re-validated, private network addresses blocked, login and CAPTCHA pages never bypassed.
- Splits combined PDFs into individual invoices and crops each receipt out of a scanned page of cash receipts.
- Skips non-accounting content and records why, so scan counts always add up.
AI classification and extraction with evidence
- Original PDFs and images go to a vision-capable model with strict structured output: first classify (invoice, credit note, statement, receipt, payment advice, not accounting), then extract supplier, number, dates, totals, tax, ship-to, line items and statement rows.
- Every extracted value must quote where it came from, and the platform checks those quotes. Answers it cannot verify are rejected rather than trusted.
- Document text is treated as untrusted data; the model has no tools and no access to the mailbox or accounting system.
- Admins choose the model per site, and every analysis is fingerprinted and audited so runs can be compared.
Deterministic rules engine
- Readiness checks for invoice number, real date, currency and a non-zero total, using exact decimal arithmetic. No floating-point money anywhere.
- Site matching: the Ship-To block wins when present, billing details are the fallback, and mismatches are held for a decision.
- Credit notes detected from headings, negative or "CR" totals, and sent as credits with the correct sign.
- Identity-based duplicate protection across copies, forwards and links, backed by a database constraint that makes a second send impossible.
- Site-specific supplier and sender exclusions, checked before analysis and again before export.
Optional human review
- Ready invoices can flow straight through; the team reviews as much or as little as it wants.
- One queue grouped by source email, with the original document side by side with the extracted fields.
- Reviewers verify, correct, retype a document, mark it "not an invoice" or reanalyse. Corrections are audited as reviewer input, never shown as source evidence.
- Bulk send for many ready invoices at once, each still checked on its own.
Accounting system delivery
- Per-site connections to the accounting system (Xero, QuickBooks or similar), sending draft bills or credits with the original PDF attached.
- Automatic contact matching, with a safe fallback when no supplier contact exists yet.
- Clear status for every invoice (pending, sent, failed, retryable), with safe retries and manual resolution.
Statement reconciliation
- Each supplier statement is broken into lines; every referenced invoice is checked in the accounting system, then searched for across the whole mailbox.
- Originals found are imported and run through the normal rules. Unmatched lines become pending invoices to request, or a placeholder bill that is flagged when the real one arrives.
- A credit-note register tracks credits on statements that are missing from the books.
Security and operations
- Role-based access per organisation and site. Public sign-up is disabled and only administrators change policy.
- Every review action, correction, send and setting change is audit-logged with before and after values.
- Evidence lives in private storage scoped to each site, with retention and legal-hold rules.
- AI requests use no-storage settings; secrets live in a cloud key vault, never in source control.
- Runs on Azure VMs, deployed with Kamal through CI over a private Tailscale network. Only HTTPS is public; the database is private and backed up nightly.
Results
From typing to verifying
The accounts team works from one queue and confirms extracted data against the original instead of keying each invoice.
Correct site, by default
Ship-To matching and per-site connections route each bill to the right entity; anything ambiguous goes to a person.
Missing invoices surface early
Statement reconciliation turns month-end detective work into a list of specific invoices to chase.
No silent duplicates or excluded bills
Duplicate and exclusion checks run before anything reaches the books, and the database enforces one send per document.
Full traceability
Every number in the books traces back to a quoted source, an email and an auditable decision.
Key takeaways
-
1
Let AI read and let code decide. Extraction is probabilistic, so site, duplicate, credit and exclusion decisions are deterministic, testable and auditable.
-
2
Make evidence a requirement. Requiring every field to quote its source made AI output checkable and gave reviewers instant context.
-
3
Build with the people who use it. Real supplier documents drove each iteration, and every edge case became a permanent test.
-
4
Fail closed. When the system is unsure, it holds the document for a decision. It never guesses its way into the ledger.
Have a process like this?
Tell us what is costing your team the most time. We will show you how we would scope it, build it and get it into production.