Document automation, Takeshi Miclat

Problem

Pulling data out of documents at scale, like vendor transaction data from an invoice or LP wire instructions from a capital notice, breaks on extraction strategy drift. Formats vary between issuers, the same entity gets referred to three different ways, currencies and units shift between documents, and every new vendor or client introduces new variations.

What I built

I started with an OCR prototype and built a layer of extraction tooling around it, each tool applying a different structural strategy.

Column-based and row-based extraction
Surrounding-text and relative-position text
Special boolean types, checkboxes and radio buttons
Classifiers at the document, page, and block level

Regex handled the first pass at variability. We then trained a machine learning model across a wide document corpus to make extraction more flexible than fixed rules allowed.

That still wasn't enough. New variations arrived constantly. New vendors, new clients, tax form revisions. Each one exposed the limits of a purely structural approach.

So I combined the structured tools with LLMs rather than replacing one with the other. A map stage handles embeddings retrieval optimized per use case through k-clustering, and a reduce stage handles augmentation and generation while preserving traceability back to the source. For documents requiring references across multiple pages, document types like credit agreements, land leases, and insurance provider agreements, we used graph retrieval augmented generation.

The system also learns document-level conventions, not just field values. Recognizing when an entire numeric column is denominated in billions, when dates follow European convention, or when a borrower appears under an alias.

On top of all of it sits a deterministic, non-AI validation layer.

Typecheck validation and standardization
Numeric operation checks
Cross-reference against external datasets

This is one key piece of my product DNA. Build the structure first. Let the model extend it, not replace it. That structure is always the core value proposition and the moat for any organization.

The decision that mattered

Clients almost always ask for confidence scores. This will invariably come up in conversations around AI. But a confidence score measures how cleanly the model chose between the alternatives it considered, and is ultimately the model grading its own work. What a client actually needs is assurance that the logic they would apply to check the answer is already in place. That's what validation does, and why it belongs in any AI system where the output gets acted on.

Traceability, type enforcement, and rule-based validation. I made sure the system had these in place for clients to truly feel they can trust the data.

That's the discipline underneath the architecture, where building the deterministic structure comes first, then having LLMs leverage them, resulting in a more consistent and trustworthy extraction. The structure is what makes unseen document types processable without becoming unpredictable.

Where this is going

A single upstream error can cascade into hundreds of validation messages, and the analyst has to trace back to the root cause, which is the same problem programmers have with compiler errors. So I designed an agentic system that uses the deterministic validation itself as context to re-assess and perform an informed re-extraction, rather than making a human parse the cascade.