The Open-Rate Drop Wasn’t Just the Subject Line
The first two emails fell from 53.2% and 46% opens to 25.6% and 27.6% after the funnel and subscriber mix changed.
When the first two emails in our newer funnel opened at 25.6% and 27.6%, I immediately wanted to rewrite them. The older versions had opened at 53.2% and 46%, so the conclusion felt obvious: the new copy was worse, the subject lines were tired, and I should begin the ancient email-marketer ritual of changing twelve things at once.
Then I looked at who was actually receiving the emails. Roughly 30% of the newer subscribers were coming through Kit recommendations. We expected those readers to be colder than people who had deliberately found us through the site, taken an action, and asked to hear more.
I was blaming the email too fast
That did not prove the copy was good. It proved I was trying to diagnose it with an average that blended different people together. If I rewrote the subject line and the open rate improved, I still would not know whether the copy helped or whether the audience mix shifted again.
The cleaner test was to duplicate the first two emails, keep the message stable, and compare cohorts by acquisition source. Now I could ask a useful question: does the same email behave differently when the subscriber arrived with a different level of intent?
Try this on your blog
- Split subscribers by where they joined.
- Send each cohort the same first two emails.
- Compare opens, clicks, and unsubscribes before changing the copy.
You’re done when: You can compare the same emails across subscriber sources before rewriting a word.
I still care about the average. I just do not let it sentence the copy before I have questioned the witnesses. Segment first, understand the audience, and then decide whether the email deserves the red pen.