Mortgage IDP cleared for auto-approval at 99.4% accuracy

Overview

A mortgage application doesn't get approved on a single document; it gets approved when 23 of them agree

An intelligent document processing agent built to handle end-to-end mortgage application processing needed to classify documents, extract fields, validate against rules, and route decisions with zero manual keying at a quality bar that auto-approval can rely on.

Firstsource enables the training-data and validation discipline behind that agent: field-extraction annotation across a diverse batch of mortgage documents, validation rule traces, an adversarial exception corpus, and straight-through-rate QA against underwriter decisions.

This was Intelligence that Operates: agent training data and quality discipline produced on the cadence the model needed, not on a research timeline.

Challenges

• Document IDP doesn't work without document-specific training data. W2s, bank statements, pay stubs, IDs, and the full range of mortgage document types each require their own field-extraction labels, validation rules, and exception coverage. Generic OCR training is not enough.
• Auto-approval depends on validation, not extraction. An agent that extracts 'income = $112K' from an application doesn't matter if the agent can't reason about that against a bank-average reading of $109K. Validation rule traces ” and the training data to learn them are where IDP agents earn the right to auto-approve.
• Adversarial documents break IDP agents that weren't trained on them. Blurry scans, rotated pages, conflicting data fields, and inconsistent formats are the production reality. An exception corpus is the difference between an IDP demo and a deployed IDP agent.

How We Made It Happen

We produced the training data and quality discipline an end-to-end Mortgage IDP agent needs to reach auto-approval-grade accuracy.

• Field-extraction labels across the full mortgage document inventory. Bounding boxes and values annotated across W2s, bank statements, pay stubs, IDs, and the rest of the mortgage document inventory.
• Validation rule traces and adversarial exception corpus. Agent logic rated for rule coverage and exception handling, with an adversarial corpus of blurry scans, rotated pages, and conflicting-data documents to harden production behavior.
• Straight-through-rate QA against underwriter decisions. Auto-approve outputs audited against actual underwriter decisions, not against a benchmark, so the agent's accuracy is the same accuracy the lender will rely on.

Conclusion

Mortgage IDP only earns auto-approval when the training data covers every document type, every validation rule, and every adversarial case the agent will see in production. Firstsource produces that data and that discipline, turning IDP into Intelligence that Operates.