How do I A/B test verification ON vs OFF in my signup funnel?

Most teams add verification to a signup funnel without measuring whether it cost them conversions. Most teams should run a proper A/B test, see the numbers, and then keep verification on with confidence. The setup is the same as any other funnel experiment, with three nuances specific to verification.

Setting up the split

Standard 50/50 user-level split, sticky to the user across visits. Half the traffic hits the form with Valid Email Checker called server-side on submit. Half hits the form with no verification. Same form UI, same downstream flow.

Treatment (verification on) should use the signup-form integration pattern: block on invalid, disabled, disposable, spamtrap; warn on risky, role, catch_all, inbox_full; pass on safe; accept on unknown. Control (verification off) accepts everything.

Metrics to measure

A surface-level signup-conversion comparison is not enough. Measure all of these:

• Form-submit completion rate. This is where verification might cost you — bad-data users get rejected and either fix their address or bounce out. Expect a 0.5 to 2% drop on the verification side.
• Confirmed signups (post any DOI step). Often the verification cohort is higher here because typos that would have failed confirmation never enter the funnel.
• 30-day bounce rate on any email sent to the cohort. Verification cohort should be much lower (often 80%+ reduction in hard bounces).
• 30-day engagement (open rate, click rate). Verification cohort is usually higher because the list is cleaner, so engagement-weighted deliverability is better.
• 90-day list quality. After three months, what fraction of the cohort is still engaged? Verification cohort holds up much better because typos and disposables never made it on.

How long to run

Two weeks minimum, four weeks ideal. Short tests catch the conversion-rate effect but miss the deliverability and engagement effects, which lag because they need a few sends to manifest. If your signup volume is high enough, a two-week test will hit statistical significance on conversion; the longer windows let you see the downstream gains clearly.

Reading the results

Almost every test we've seen at Valid Email Checker customers comes out the same way:

• Verification cohort sees a small drop in raw form submits (0.5 to 2%).
• Verification cohort sees a meaningful drop in bounce rate on the first campaign sent (often a third or less of the control bounce rate).
• Verification cohort sees stable or slightly higher engagement.
• Verification cohort holds a larger active-subscriber base at 90 days than control, despite the smaller raw signup count.

The math is consistent: verification trades a small loss of bad signups for a much larger gain in list quality. The exception is funnels where the rejected signups were already mostly bots and bad data — there verification looks neutral on conversion and pure win on quality.

When the test says verification is a loss

If your test comes back with verification hurting both conversion and engagement, double-check the implementation. Common mistakes: blocking on too many statuses (don't block catch_all or role at the form — warn instead), no timeout fallback on the API call (a slow VEC response shouldn't fail the form), or treating unknown as a block (it's auto-refunded and inconclusive, not a bad-address signal).