How to choose a fraud score threshold

Anyone who answers this with a single number is guessing on your behalf. The score is the same in every flow; what changes is what a mistake costs. Wrongly blocking a newsletter signup costs you an email address. Wrongly blocking a password reset costs you a customer who cannot get into an account they already pay for. Those two cannot share a cutoff, and the reason people end up with a bad threshold is almost always that they picked one number and applied it everywhere.

1. 1

   Work out the exchange rate before you pick a number

   Every threshold is a trade at some rate: how many real customers you will turn away to stop one fraudulent signup. You do not need a precise figure, but you do need to know which way it leans, because that decides everything downstream. Protecting a payout or a credit balance justifies refusing a lot of good traffic to stop one bad transaction. Protecting a free newsletter almost never does. Write the two costs down in whatever unit is honest for your business, and the rough shape of the threshold falls out of the comparison.

2. 2

   Look at your own score distribution before you set a cutoff

   A threshold chosen before you have seen your traffic is a number pulled from someone else's business. Run scoring in log-only mode for a week or two — score every signup, act on none of it — then plot how the scores fall. Most products find the bulk of real traffic clustered low with a thin tail above it, and the useful cutoff is where that tail separates rather than at any round number. If your distribution has no separation at all, that is worth knowing before you start blocking on it, and no amount of threshold tuning will fix it.

   // Log-only: score everything, block nothing, decide later.
   const r = await score({ ip, email, phone });
   await log.signup({ userId, score: r.score, verdict: r.verdict });
   return createAccount();  // no branch on the score yet

3. 3

   Use three bands, not two

   The instinct is one line: above it refuse, below it allow. That framing forces every borderline case into one of two wrong answers, and borderline is exactly where real customers and real attackers overlap. A middle band routes ambiguity to a step-up — an emailed code, an SMS, a card check — which is the only move that is cheap when you are right and recoverable when you are wrong. A real customer completes a one-time code in a few seconds. Widening the middle band is nearly always better than agonising over where a single line goes.

   const r = await score({ ip, email, phone });
   
   if (r.verdict === "block")  return reject();
   if (r.verdict === "review") return sendOneTimeCode();  // the band that saves you
   return createAccount();

4. 4

   Set the level per flow, using one score

   Score once, then let each flow read the same number against its own cutoffs. LayerCall exposes this as a strictness level on the request rather than making you hard-code numbers: level 0 blocks at 85 and reviews at 55, level 1 — the default — blocks at 70 and reviews at 40, level 2 blocks at 55 and reviews at 30, level 3 blocks at 40 and reviews at 20. The score itself is untouched; only the two boundaries move. A sensible starting arrangement is the lenient end for anything a real customer does while already logged in or already known to you, the default for new signups, and the strict end reserved for the flows where money moves.

   # same score, different boundaries per flow
   curl "https://www.layercall.com/v1/score/ip?ip=1.2.3.4&strictness=0" \
     -H "X-Api-Key: YOUR_KEY"   # password reset — protect the existing customer
   
   curl "https://www.layercall.com/v1/score/ip?ip=1.2.3.4&strictness=2" \
     -H "X-Api-Key: YOUR_KEY"   # payout request — protect the money

5. 5

   Measure both mistakes, or you will only ever tighten

   Fraud that gets through is loud: it arrives as a chargeback, a complaint, a cleanup job. A customer wrongly refused is silent — they close the tab, and nothing in your dashboard records that they existed. If the only feedback reaching you is the loud kind, every review will conclude the threshold is too loose, and it will ratchet in one direction for as long as the product lives. Counter it deliberately: count challenges issued and how many were completed successfully, because a step-up that real people pass at a high rate is a band doing its job, and one they abandon is a band set too wide. Feed confirmed outcomes back so the question stops being a matter of opinion.

   curl -X POST "https://www.layercall.com/v1/outcome" \
     -H "X-Api-Key: YOUR_KEY" -H "Content-Type: application/json" \
     -d '{ "request_id": "req_...", "outcome": "legitimate" }'

6. 6

   Change it on evidence, and change one thing at a time

   The worst moment to redesign a threshold is the morning after an incident, when the only visible cost is the one that just happened. Move it deliberately instead: one level at a time, then watch both numbers — signups refused and challenges passed — for long enough to see a weekly cycle, since traffic on a Sunday does not look like traffic on a Tuesday. If you change the strictness level and the scoring inputs in the same week, you will not be able to attribute the result to either.