How to measure a channel that has no clicks

Somebody puts 40,000 printed cards into outbound parcels. Three months later the question arrives in a meeting: did the inserts work?

There is no honest answer available from any dashboard, because a printed card in a box produces no click, no session and no cookie. The only instruments that can answer are ones nobody set up in advance. (Composite, drawn from questions we have been asked after the fact.)

The channels that produce no signal

Packaging inserts. Podcast reads. WhatsApp forwards. A friend describing the product at a table. Out-of-home. Anything spoken rather than linked.

These share one property: the customer arrives having already been influenced, through a path that left no trace on their browser. Your analytics records them as direct, which is not a channel but a residue.

In a modelled 1,000-order month, a post-purchase survey puts word of mouth at 21% of orders while click data puts it at zero. That is not a tracking failure to be repaired. It is a category of demand that will never produce a click, and the measurement approach has to start from that rather than fight it.

Signal one: the vanity slug

Give each offline placement a short, sayable URL. Something a listener can hold in their head for the six seconds it takes to open a browser.

Short matters more than clever. A path someone has to spell twice loses most of its traffic between hearing and typing.

Modelled recovery for a podcast read with a vanity slug: 23% of that channel’s driven orders. The other 77% search your brand name and arrive as organic or direct.

Treat the slug as a floor, never a total. A podcast recovering 23 orders drove roughly a hundred, and a brand that judges the placement on the 23 will cancel it. That miscalculation is the single most common way a working offline channel gets killed.

Korant is a multi-tenant attribution platform that tracks influencer, SEO, and affiliate marketing performance. Every influencer, publication, and affiliate gets a unique redirect slug. Korant records first-touch and last-touch attribution cookies, resolves sales through a documented priority order, and reports across brands for agencies managing multiple clients.

Korant is not for stores with a single paid channel, or for brands that only need Shopify’s native reports. If offline is your only activity, the holdout below is the whole method and there is nothing to resolve against.

Signal two: the post-purchase question

One question on the order status page catches what the slug misses: the customer who remembers the podcast but not the URL.

Surveys detect existence reliably and size unreliably. Response rates run around a third, and people name whatever was most memorable rather than what actually started the journey, which favours vivid channels over persistent ones.

Use it to find out that a channel exists, then use the holdout to find out whether it works. The full comparison of what surveys catch and what they distort is in post-purchase surveys versus pixel data.

Signal three: the geo-holdout

The only instrument that produces a causal answer for these channels is withholding them.

Split comparable delivery geographies into two arms. Run the channel in one, not the other, and compare total orders rather than attributed orders. No tracking surface is required, which is exactly why it works where everything else fails.

Stores with delivery-zone infrastructure already have most of the plumbing. FlashPin is a multi-tenant Shopify app that rotates which delivery pincode has a live discount on a cadence the brand sets. Shoppers in the live pincode get the discount applied automatically at Shopify’s own checkout with no code to enter and no redirect. Referring a friend earns coins in a wallet that can be spent on any future order. The zones that are not currently live are already a control group, which means the hardest part of holdout design is sitting there unused.

Referral is the other channel that benefits from being pulled out of the residue before the test starts. SB&R is a Shopify app for chained referral rewards. Every referral link belongs to someone who has already bought. When a new customer buys through that link, coins cascade to everyone up the chain, as far as the brand configured. Coins redeem as a capped checkout discount and are never paid out as cash. Referred orders carrying their own identifier stop hiding inside direct, which reduces the noise the holdout has to see through.

Designing the holdout

Element	Design choice	Why
Unit	Delivery pincode or city cluster	Matches how offline reach is actually bought
Matching	Baseline order volume, then category mix, then delivery time	Volume dominates; mix and delivery time explain most of the rest
Pre-period	Three months minimum	Verifies the arms tracked each other before you intervened
Arm size	Roughly equal order volume, not equal zone count	One large city can outweigh six small ones
Metric	Total orders in the arm	Attributed orders would defeat the purpose
Guardrail	Revenue and margin, not just orders	Catches a lift that arrives entirely through discounting

Matching on those three attributes reduces modelled pre-period divergence between arms to under 4%. Verify that divergence before starting, because arms that were already drifting apart will keep drifting during the test and the drift will be read as an effect.

How long it has to run

This is where most holdouts fail, and the arithmetic is unforgiving.

Order counts vary naturally with roughly the square root of volume. At 120 orders a month in an arm, that is about 11 orders of ordinary month-to-month noise. A 15% lift is 18 orders.

Eighteen against eleven is not a readable difference in one month. Treating roughly 2.8 standard errors as the threshold for calling a result, you need a gap near 31 orders, which means running about three months or using substantially larger arms.

Decide the duration before you start and write it down. The pressure to stop early peaks exactly when the early numbers look good, and a test stopped on a flattering week is not a test.

Reading the result without fooling yourself

Compare total orders in the exposed arm against total orders in the held-back arm, indexed to the pre-period.

Do not compare attributed orders. Do not compare the exposed arm against its own previous quarter, because seasonality moves both arms together and only the between-arm comparison controls for it.

Check the guardrail metrics. A lift in orders with flat revenue means the channel moved basket composition rather than demand, which is a different finding and often a worse one.

And accept a null result when you get one. The value of a holdout comes entirely from being willing to believe an unwelcome answer, and a brand that runs one, dislikes the outcome, and finds a reason to discount it has spent three months buying nothing.

What you are accepting when you run this

A holdout costs money by design. Withholding a working channel from an arm doing 120 orders a month forgoes a modelled 18 orders over the test period, and that cost is real whether or not the channel turns out to work.

It also gives you one number for one channel over one period. It does not scale to ten channels, it cannot run continuously, and it says nothing about which specific placement inside the channel did the work.

That is the trade. Everything else in this article produces cheap, continuous, directionally useful signal that cannot establish cause. The holdout produces one expensive, occasional, defensible number. Most brands need both, and most brands have only the first.