Post-opening validation: how to know if a site selection model was right

Most expansion teams put real effort into the decision and almost none into the postmortem. A site gets scored, a forecast gets signed off, the lease gets approved, and attention moves to the next deal. A year later, nobody goes back to ask whether the number the model produced came anywhere near the truth.

That gap costs money. A model nobody checks cannot improve, and it drifts quietly as markets, competitors, and your own network shift around it. Validation closes the loop and gives you grounds to defend a scorecard with evidence rather than habit.

Closing the loop takes discipline more than new math. The prediction is already on record. Once the site has traded long enough to mean something, you set it next to reality and write down what you learned.

• Forecast versus actual is the headline comparison. Put the predicted first-year sales or volume next to what the site really did, and read the signed gap, which tells you the direction of the error, hot or cold, on top of its size.
• Mean absolute percentage error, or MAPE, collapses the gap into one comparable number. Tracked across a cohort of openings, it shows how accurate the model is on average and whether recalibration is moving that accuracy.
• Ramp versus predicted matters because new stores do not open at maturity. Compare the actual month-by-month ramp to the curve you assumed, since a site can reach its mature number while taking far longer or shorter to get there, and that timing reshapes the pro forma.
• Score versus performance steps back from the dollar figure to ask whether the explainable site score ranked locations in the right order. When the high-scoring sites outperform the low-scoring ones, the model is sorting deals well even where a single point estimate misses.

When you measure matters as much as what you measure. The first weeks of any opening run hot, with a grand opening, local press, curiosity traffic, and a promotion calendar that flatters the totals. Judge a site on that window and every location reads like a winner.

Wait for the opening rush to fade and the site to settle into a steady run rate. A trailing window of stabilized weeks, well after launch, gives a fair read on the mature number the forecast was trying to predict. Apply the same window to every site so comparisons stay consistent, and keep the opening period as a separate ramp observation instead of folding it into the verdict.

One opening proves little on its own. A group of them can prove a pattern. Group your openings by vintage, by format, or by market type, and watch how error trends across the set. A single site that missed badly might be a fluke, while ten sites that all came in under forecast is something the model needs to absorb.

Watch the analogs closely. Most forecasts lean on comparable existing locations to predict a new one. After opening, check whether the analogs you chose actually behaved like the new site. Where they did, lean on that analog set harder. Where they did not, the matching logic needs work, well beyond the weights.

Validation only pays off when it feeds back into the model. Once you can see where the misses cluster, three levers are available, and the data points to whichever one is loose.

• Weights come first when high-scoring sites underperform low-scoring ones. Adjust how much each component of the score counts until the ranking matches what the cohort actually did.
• Analogs are the issue when the comparable sites you leaned on did not behave like the new opening. Tighten the matching with a different trade-area shape, different competition, different demographics, or a cleaner set of peers.
• Assumptions are the fix when the mature number came in close but the timing was off. Repair the ramp curve and the seasonality assumptions rather than the site model, since the trade area was often right and only the calendar was wrong.

Recalibrate deliberately, one lever at a time, so you can tell what actually moved the accuracy. A model that swings hard after every opening is overfitting to the last data point rather than learning from the cohort.

Every recalibration should leave a paper trail. Keep a decision record for each validated site: the prediction, the actual, the error, and the specific change you made to the model in response. Note the comparison window and the data vintages so the next person can reproduce the read instead of relitigating it.

Documentation is also where confidence comes from. When a committee asks why the model deserves trust, the answer stops being a feeling and becomes a history of forecasts checked against actuals and weights adjusted on evidence. A site brief lands harder when the model behind it carries a track record you can point to.

Treat every miss as raw material. A gap between forecast and actual is the clearest signal you have about what the model failed to see. The teams that end up with trustworthy models treat each gap as a question worth answering about what got overlooked and how to teach the model to catch that pattern next time.

Run validation on a steady cadence, fold the lessons back into the weights and analogs, and the error trend bends in the right direction over time. Accuracy earned opening by opening is what separates a model people quietly distrust from one they will back with real capital.