Measuring It: Did the Copy Actually Work?

You can write the best society email your club has ever sent and have no idea it worked. Or you can write a dud and feel brilliant about it, because the open rate looked lovely. Both happen all the time. This section is about telling the difference.

Here's the trap most clubs fall into: they measure the wrong thing, celebrate it, and learn nothing. So before we get to A/B tests and readability scores, let's fix the scoreboard.

Open rate is a liar now

Stop trusting it. In September 2021 Apple rolled out Mail Privacy Protection, which pre-fetches the tracking pixel the moment an email lands, whether or not a single human ever lays eyes on it. Apple Mail is roughly 58% of all opens globally (Litmus). On a typical members' list, stuffed with iPhones, your open rate is inflated by 15-35% of pure machine noise.

So when your monthly newsletter reports a 62% open, a big chunk of that "62" is a robot in Cupertino fetching an image. It read nothing. It booked nothing. It is not a person who wants a tee time.

Open rate isn't useless. It's diagnostic colour, the starting gun, a rough signal that your subject line and sender name didn't get binned. But it is never the verdict. Treat it like the temperature outside: worth a glance, not a reason to cancel the round.

What "good" actually looks like

Good is bookings. Enquiries. Society deposits. Membership applications. Tee times reserved. The only numbers that prove your copy worked are the ones tied to money changing hands.

An email with a 48% open and 11 society enquiries beats an email with a 62% open and 2 enquiries every single time. It isn't close. If you remember one thing from this whole section, make it that.

The honest way to judge a send is a ladder. Climb it in order:

Delivered. Did it even land? ~98.6% is healthy in hospitality. Below that, fix deliverability before you touch the copy.
CTOR (click-to-open rate). Of the humans who opened, how many clicked? Median is 6.81%; a strong club send clears 12%. This is your new headline metric, because it measures what happened after a real person read the thing.
Click rate. Raw appetite to act. Median ~2.3%; 4-6% is top-tier.
Bookings / enquiries / conversions. The only metric that pays the wages.
Unsubscribe rate. The cost of the send. Keep it under 0.2% per campaign.

CTOR is the clever one. It strips out the deliverability and subject-line question (because everyone in the calculation already opened) and leaves you staring at exactly one thing: was the copy and the call-to-action any good? That's craft, measured. In plain terms: 500 people open the newsletter, 40 click through to the booking page, that's an 8% CTOR. Above median. Your words are pulling their weight even if the headline open number looks ordinary.

One more rescue trick. For the slice of your list on Gmail and Outlook, the non-Apple crowd, open rate is still real. Calculate opens and CTOR for that segment alone and use it as your directional benchmark. Ignore the Apple-inflated portion when you're making decisions.

Write the target before you press send

The fastest way to stop kidding yourself is to commit to a number before the email goes out. Not after, when you can pick whichever stat flatters you.

So write the line down. "This society email succeeds if it produces 8+ enquiries and 3+ booked dates." Then judge the result against that, full stop. Post-send rationalising is how a 2-enquiry email gets called a win because "the open rate was strong."

Pick the right benchmark too. Hospitality opens sit around 25-28%, and a one-off campaign runs roughly 2.37% CTR (Stripo). But automated, recurring flows, your welcome sequence, your post-visit follow-up, can clear 15%+ CTR. So a monthly broadcast at 2% CTR is normal and fine. A welcome automation at 2% is broken. Judge each send against the benchmark for that kind of send, not one blanket number you read on a blog once.

How to run an A/B test without fooling yourself

Two rules. Break either one and you've learned nothing.

Rule one: change exactly one thing. The entire logic of an A/B test rests on attribution. If you change the subject line and the CTA and the hero image in the same test, and version B wins, you cannot say which change did it. You've spent your test and bought yourself a shrug. So isolate a single variable. Subject A vs subject B. Or headline A vs headline B. Never both at once.

Rule two: respect the maths, or build patterns instead. The textbook says aim for 95% confidence before you crown a winner, and detecting a 20% lift on a 40% open rate needs roughly 592 subscribers per variant. Here's the honest bit most agencies won't tell a club: you probably don't have the list for that. A club with 900 engaged contacts splitting 50/50 gives ~450 per variant. Below the line. One single send will almost never reach significance.

That's not a reason to skip testing. It's a reason to test differently.

Log the direction, every time, in a running sheet. Did the specific date beat the vague one? Tick. Did the action CTA beat "Find out more"? Tick. After 6-10 sends, the consistent winners, "specific dates always beat vague," "scarcity always lifts clicks", become reliable house rules even without per-test significance. Patterns over time beat one heroic test you can't trust anyway.

The mechanics, when you do run one:

Split the list randomly: Group A (control), Group B (one change).
Send to a test slice, measure for 24-48 hours (never under an hour, never over seven days, most action lands within hours but late openers and time zones need room).
Roll the winner out to the rest.

What to actually test

Subject lines first. It's the highest-leverage test you own, because nothing else matters if the email is never opened. Try:

Specific vs vague: "Sat 12 July: 3 fourball slots left" vs "July tee times now open"
Benefit vs curiosity
With member's name vs without
Short (under 40 characters) vs long

A clean one looks like this. Variant A "July tee times now available" vs Variant B "Sat 12 July: only 3 fourball slots left." Same body, same send time, 50/50 split, measured at 24 hours. The only difference is the specificity and scarcity in B, so any lift is attributable. That's the whole point.

CTAs next, measured on click and CTOR, never on opens:

Action-specific vs generic: "Book your tee time" vs "Find out more"
First vs second person: "Reserve my spot" vs "Reserve your spot"
Single CTA vs several
Button vs text link

"Book your tee time" usually thrashes "Find out more" because it tells the member exactly what happens next. When the data confirms that, you're watching a copywriting principle, one clear next action, prove itself in numbers.

Headline vs headline, in the body. Same email, two opening headlines, measured on click and downstream booking. This is where copy craft shows up cleanest, because the subject line and deliverability are held constant. Pure words against pure words.

Readability is a conversion lever

Members read on phones, in the car park, half-watching the telly. Hard copy gets skimmed, misread, or abandoned before the CTA. That's revenue walking out the door because a sentence had a comma too many.

Aim for a Flesch Reading Ease score of 60-70, roughly 8th-grade level (Yoast). Around half of UK and US adults read at or below grade 8, so this isn't dumbing down, it's hitting the room. Keep most sentences under 20 words; the Plain English Campaign targets an average of 15-20 (Plain English Campaign). Pick the short word: help over facilitate, buy over purchase, book over secure your reservation.

Watch it work:

"We are delighted to extend an invitation for you to participate in our forthcoming members' open day, which will be held on the premises."

28 words. Latinate sludge. Becomes:

"Join us for the members' open day. It's here at the club on Saturday 12 July."

Two short sentences. Plainer words. Reads in one breath each. Same information, a fraction of the effort to act.

The read-aloud test

This is the most powerful proofreading trick there is, and it's free. Before any email ships, read it out loud.

The ear catches what the eye glides past. Take this draft line: "Secure your reservation for the inaugural twilight competition series commencing in early autumn." On screen you skim it. Out loud, you run out of breath and trip on "inaugural" and "commencing." The ear flags it instantly. Rewrite: "Book your spot in our new twilight series. It starts in early autumn."

That's the breath test: if you can't read a sentence aloud in one comfortable breath, it needs a full stop or a comma. It's a tool-free proxy for sentence length any club marketer can use, no software, no score, just your own lungs.

Tie it back to the craft

A dashboard tells you that a variant won. Only the copywriter can tell you why, and the why is the part that compounds.

A subject line won because it named a specific date. A CTA won because it promised one clear next action. Those aren't trivia, they're house rules, and over a season they build into a club voice that converts. Numbers without interpretation are just trivia. Interpretation without numbers is just opinion. You need both, and you, the person who wrote the words, are the only one who can supply the second half.

Quick-reference checklist

Before you send:

Written the success target for the metric that matters (e.g. "8+ society enquiries")
Picked the right benchmark for this type of send (broadcast vs automation)
Testing exactly ONE variable, if testing at all
Run the draft through a Flesch tool, aiming for 60-70
Split any sentence over 20 words
Swapped Latinate words for plain ones
Read the whole thing aloud and cut every stumble
Paragraphs kept to 2-3 sentences for phone scanning

After you send, judge in this order:

Delivered (~98.6% healthy)
CTOR (median 6.81%, strong 12%+)
Click rate (median ~2.3%, top-tier 4-6%)
Bookings / enquiries: the only one that pays the wages
Unsubscribe rate (keep under 0.2%)
Logged the test direction in your running sheet

The rules to tattoo on your forearm:

Open rate is a liar; bookings are the truth.
One variable per test, or learn nothing.
One club, one test rarely hits significance, so build patterns over many sends.
If you can't say it in one breath, it's too long.
The dashboard says what; only you can say why.