How Do You Measure Progress of Something You Cannot See?

There's a moment most practitioners know well. The workshop went well, the participants were engaged, and the debrief surfaced real issues. Decision makers left the room talking differently than when they walked in.

But at some point, someone asks the tough question "how do we know this is working?"

Or another variation might be "how do we know if this training will stick?"

Not "did people like it", Nor is it "did it feel meaningful to the participants",... but is the organization actually becoming what the training was intending to produce? Worse yet, the presumption that this training may go like the other ones, and wonder if any of it will remain alive in the organization.

For most practitioners, that question lands somewhere between uncomfortable and unanswerable.

Most leadership and organizational development work is thoughtful, grounded, and well-intentioned, so the intent and effort is usually not the issue. In fact, most challenges in just about any type of training are systemic in nature be it in specific skill training, or more qualitative work alike.

The qualitative field has invested heavily in frameworks for doing the work, and almost nothing in tools for measuring whether the work is producing durable change. Where skill-based trainings tend to have quantifiable means of measurement, the more qualitative fields of leadership development and organizational development have very little ways to meaningfully measure results.

So what gets measured instead? Participation rates, satisfaction scores, and anecdotal evidence from debrief conversations.  Sometimes a pre/post engagement survey, though these tend to measure awareness of concepts rather than shifts in underlying capacity.

These all tend to live at the events level of awareness, which only focuses on what happened in that moment the data was collected.  Very few get to the behavioral pattern level of awareness, which tracks patterns over time of leading indicators of behavior centered on the capacity practitioners are trying to build.  Behavioral patterns are more difficult to track for that exact reason: they rely on placeholders for events that are themselves placeholders for something really tricky to measure.

Now imagine being to asked to measure progress on leadership and/or organizational development work from the structural roots of the problem.  This is exceptionally tricky to do, which is likely why no reliable gold standard of measurement exists for it.

Frankly speaking, I think the struggle with measurement is also compounded by lack of attempts to so.  Quantitative professionals dismiss the idea as possible due to its reliability challenges and difficulty.  Qualitative professionals dismiss the idea because the complexity of human nature is perceived as a sacred cow that cannot be calculated.I call B.S. on both fronts.

Without really trying, we cannot make the determination this is not possible.  Half or more of the challenge, to me at least, appears to be grounded on effort.  Not enough people on either side of the quant/qual side of the bridge are doing a very good job working together.  If only that were not the case, who knows what the free flow of information could be!Beyond the effort challenge, there is a very real challenge with reliability.  Assuming the creative capacity is there, the amount of trial and error can be daunting, as well as extremely time consuming.  Any effort tried may prove to produce different results each time, which will be branded (and rightfully so) as unreliable in its current form.

But these are not good enough reasons to give up on trying to measure. 

The practitioners who have yet to know how measure these things are flying semi-blind.  Those making consequential decisions about where to focus, how long to stay, when to declare progress are all without the prerequisite information those decisions probably need.  So when results do not materialize, or when a crisis exposes that nothing has actually changed at the capacity level, there is no data available to explain why.

The measurement gap will only continue to grow as organizations are allocating real resources to this kind of qualitative work.  The leadership development budgets, culture initiatives, and transformation efforts are primarily measured through story and human instinct.

Mind you, that’s a perfectly reasonable starting point.  But it is an unreasonable endpoint if we can truly try to measure these engagements over time.

What the field needs is a way to quantify where an organization's capacity actually stands, identify which dimension is the binding constraint, and track whether interventions are producing genuine change over time.   Certainty not perfectly, and not with false precision, but rather with enough structure to move from instinct to evidence.  The point is to produce useful insight over precise forecasting.

That kind of measurement is harder to build than a framework. It requires understanding how these capacities interact, how they decay, and what conditions allow them to grow. It requires distinguishing between surface compliance and durable change. And it requires honesty about what the data can and cannot yet tell us.

The practitioners doing this work deserve better tools. The organizations investing in it deserve better answers. Building the bridge between the rigor the problem requires and the conditions under which real change actually happens is among the most important work in the field right now.