Marketing measurement has gotten good at telling us what happened after we spent the money. We know where an ad ran, what it cost, who saw it, what they clicked, whether they bought something. We feed that information back into the plan and keep optimizing.
We are worse at answering a more basic question before the money goes out:
Was the creative worth putting money behind in the first place?
We spend weeks or months developing creative. It gets reviewed, revised, approved, tested, revised again, and handed to media. Then we put real money behind it and finally see how people react. At that point, the test has already gotten expensive.
Gary Vaynerchuk has been making this argument from the creative and media side for years. His position is direct: “I think it is insane to spend [even] $1 of working media on creative that has not been validated on social.” He has also argued that too much marketing measurement is “predicated on maybe or potentially vs. actual,” and that bringing creative and media back together pushes the work closer to real business results.
The measurement implication is what interests me. When creative and media split, creative lost one of the most useful things measurement can give it: a live feedback loop.
Put the test before the spend
Digital media gave us an incredible amount of information. We see delivery and response almost immediately. We run experiments. We change audiences, placements, frequency, spend, and dozens of other variables while a campaign is live. That is progress, but it also created an imbalance. We measure how we distribute an ad in incredible detail while doing much less to validate whether people want the ad in the first place.
You can build a sophisticated media plan around mediocre creative. You can optimize its delivery and build a dashboard explaining exactly what happened. The dashboard doesn’t fix the creative.
Gary’s model for organic social moves the learning earlier. Make a lot of creative, put it into the market, see what gets a reaction, then put working media behind what has already shown it can earn attention. Organic performance isn’t proof that something will drive incremental sales, but it gives you information before you’ve committed the bigger budget. You learn while you can still do something with the information.
That connects directly to Velocity. My argument there was that analytics should shorten the distance between useful information and the next decision. Skipping validation so you can get into paid media faster isn’t velocity. You’ve just moved the learning to a more expensive part of the process.
I’ve watched this happen more than once. Performance starts to slow, the team decides the creative is wearing out, and there isn’t a validated replacement ready. Tired creative stays in market while the next round gets developed and approved, and then we spend money finding out whether that round works too.
The problem isn’t that we won’t know everything ahead of time. We won’t. The problem is that we could have known something sooner.
The test doesn’t have to live on a screen
Gary has also been talking about a barbell forming in marketing, with extreme digital on one side and extreme analog on the other. He has talked about how those sides feed each other too: physical experiences create reactions and content that can travel digitally.
From a measurement perspective, there are two different things happening on the physical side. One of them is a fast feedback loop.
An event, pop-up, sampling activation, run club, collectible, or product drop puts an idea in front of real people. Did they stop? Did they participate? Did they bring someone else over? Did someone pull out a phone and post without being asked? Did creators pick it up? Did something that happened with 300 people become something hundreds of thousands wanted to watch?
Those are reactions to creative, and you can get them quickly. The event isn’t automatically the slow, immeasurable side of marketing. It can be the test. The reaction creates content, that content gets another test through organic distribution, and what keeps working earns more investment.
That is a functioning feedback loop. But it isn’t the only value the experience creates.
Some of the value stays in the room
If the only value of an event is the content it produces, we’ve reduced the analog side of Gary’s barbell to another media input. The event matters because it creates content. The collectible matters because people post it. The community matters because it generates reach.
That can’t be the whole story.
Sometimes the experience matters because of what happened there. Someone showed up and met people. They got something they cared about. They became part of a group. Maybe they came back. Maybe they kept what they bought for ten years. Maybe the brand became connected to a period of their life.
None of that requires an Instagram post to have value.
There are two pieces here. One is catalytic. The experience creates a reaction, that reaction creates content, and the content gives us another chance to test the idea. We can get feedback on that fast.
The other piece is harder. Sometimes being there mattered. Belonging mattered. Owning the thing mattered. That value is intrinsic to the experience, and it doesn’t necessarily show itself on the same timeline.
Treating those as the same measurement problem is where we get into trouble.
Some things don’t resolve into a clean number
We can look for signals. Do people come back? Does retention change? Do brand metrics move? Does sales behavior change? For a collectible, maybe secondary-market activity tells us something. For a community, maybe participation over time does.
All of that is useful, but it doesn’t necessarily prove the full value. Analytics has a tendency to keep looking until we find a metric and then feel like we’ve solved the problem. Sometimes we have. Sometimes we’ve just found the thing that was easiest to count.
Someone feeling like they belong to something is real. Turning that feeling into an incremental ROI number is a different problem.
There is some irony in this because even the parts of advertising we treat as highly measurable aren’t always as measurable as they look. Randall Lewis and Justin Rao studied 25 large advertising field experiments involving millions of customers. Even at that scale, the median confidence interval around advertising ROI ran more than 100 percentage points wide. Advertising effects are hard to estimate precisely, even with randomized experiments and enormous samples.
A dashboard can show a number to two decimal places. That doesn’t mean we know the true effect to two decimal places. We already accept a lot of uncertainty in marketing. We just feel better about it when the metric updates every morning.
Then we get to community, experience, brand meaning, or identity and suddenly want a perfect causal chain before we’re willing to defend the investment. That’s where measurement starts creating its own bias. The things that are easier to count become easier to fund, even when ease of measurement and value aren’t the same thing.
Not everything runs on the same clock
This isn’t an excuse for lazy measurement. If we can design a better experiment, we should. If we can tighten the feedback loop, we should. “We can’t measure it” can’t become a shield for work that simply hasn’t been measured well enough.
But the opposite is a problem too.
If you’re building a community, you don’t get to decide after two events whether it worked. If you’re creating a collectible people care about, a seven-day conversion window won’t give you the answer. If people start folding your brand into their identity, you’re dealing with something that can take years to build.
We should still look for evidence. Brand tracking can tell us something. Retention can tell us something. Long-term sales behavior can tell us something. A causal experiment can give us stronger evidence about a specific effect. Those are all useful inputs into the decision.
They still may not capture the whole thing.
That’s the part I don’t want analytics to paper over. If the argument is that some value develops differently and on a different clock, the answer can’t always be to invent another KPI and pretend we’ve made the uncertainty go away.
There may not be a KPI for belonging. Part of running a business is making decisions under uncertainty.
Measurement should help us make the decision
That’s the part I would add to Velocity now.
The goal is still to get useful information into the decision faster. If something can give us a useful answer tomorrow, waiting six months is bad analytics. If something takes years to build, demanding a seven-day answer is bad analytics too.
Those aren’t the same problem, and they shouldn’t be treated like they are.
I want creative tested before we pour working media behind it. I want measurement involved early enough to change the decision, not just document what happened afterward. Wherever we can learn faster, we should.
But I also want an organization capable of saying, “We don’t know exactly what this is worth yet, but we believe it’s worth building.”
That’s harder to put into a budget deck. It’s also sometimes the more honest answer.
Measurement should tell us what we know, what we don’t know, and where we can learn faster. It should make the next decision better. What it shouldn’t do is turn uncertainty into a reason not to act.
Because once the only things we’re willing to build are the things we can prove quickly, measurement isn’t informing the strategy anymore.
It’s writing it.