Real campaigns.
Real numbers.

Why does manual A/B testing waste sends in cold email?

Manual A/B testing wastes sends because every email gets equal traffic until you manually intervene — and by then, the weaker emails have already consumed hundreds of leads. Before Apex Overlay, the team ran a standard Instantly A/B test across three subject lines on a list of 1,053 HR leads. All three emails received equal traffic. The plan was to monitor performance over a few days, pick the winner, and manually shift volume — the way most teams do it.

The issue isn't that the approach is wrong. It's that it's slow. While you're waiting for enough data to feel confident, the weaker emails keep consuming sends. At typical cold email volumes, that delay costs you measurable pipeline — not in theory, but in the actual number of clicks and replies you didn't get.

There's a second problem Instantly doesn't surface: not all clicks are human. Email security systems — common in HR and enterprise environments — automatically scan links in incoming emails, creating fake clicks that look identical to real ones. Instantly's reported click rate was 7.88%. After filtering out bot clicks (based on how fast they happened), the real human click rate was 5.41%. The A/B test was partly running on noise.

How much does Apex Overlay improve cold email reply rates?

In this campaign, Apex Overlay delivered +5.5% more engagement than Instantly’s native A/B testing by routing 95.7% of sends to the winning email within 12 days. The engine ran using the same three subject lines. Rather than holding equal split until you have enough data, the engine adjusted how often each email was sent after every reply, click, and open — using real performance data to figure out which email was working best.

By days 4–7, one email had pulled ahead. The engine began sending more of it automatically — no manual intervention, no dashboard check required. By day 12, 95.7% of sends were going to the winning email. The other two weren't killed — they kept getting a small share of sends to keep the sending pattern looking natural to spam filters.

Which job titles respond best to cold email in HR outreach?

Leadership-level Operations and HR contacts responded best, with click rates of 8.4% and 7.6% respectively. The breakdown split the 1,053 leads into five groups by seniority and job function. The curiosity-style subject line won across leadership job titles. Only Management-level Operations contacts responded better to a direct pitch — a finding that would have been invisible in a standard A/B test.

How fast can automated optimization find your best-performing email?

In this campaign, Apex Overlay identified the best email within hours and generated a qualified reply on day 1 — before most A/B tests have enough data to start counting. Standard A/B testing says you need enough sends per email before the data is meaningful. At typical cold email volumes that means days — sometimes a week or more — of equal-split sending before you make a change. During that time, every email gets the same share of sends regardless of what the early results say.

The Notetaker campaign showed what happens when you don't wait. Within a few hours of the first sends going out, the optimization engine had already started sending more of the better-performing email. By the end of day 1, 49% of sends were going to the best email. By day 3 it was 76.8%.

That shift happened fast enough to matter: a positive reply arrived within 3 hours of the first optimised send. That's a qualified opportunity the campaign would likely have missed — or delayed — under equal-split testing.

What subject line styles work best for HR cold email?

Low-pressure, curiosity-driven subject lines outperformed direct pitches in HR outreach. "Thought this might be useful" — a soft, deliberately non-salesy subject line — outperformed both a regulation-angle subject and a pain-point subject from the first sends. This is a pattern the StaffCircle data also surfaced: in HR outreach, low-pressure framing tends to outperform direct problem-statement framing.

The winning email is currently generating a 3.2% reply rate. The other two emails sit at 0%. Without optimization, 23% of sends would still be going to those underperforming emails daily — burning budget on copy that isn't working.