Speaking Data: L&D Impact Measurement Strategy
Watershed's annual report on measuring the impact of learning is a good way to explore challenges, gaps, and opportunities in L&D [1].
Enhance your L&D strategy with our latest industry report, Measuring the Business Impact of Learning 2024. It offers a detailed analysis of current trends, challenges, and best practices for effectively measuring the impact of learning initiatives.
After analyzing the report and the supporting data findings, here's my summary of the highlights. Let's start with the sad news:
Of those surveyed for our report, 56% said their L&D function could gauge the business impact of learning. However, the strategy for the remaining respondents is less promising.
Remember, every self-reported survey result is much more likely positive than reality (selection bias, recency bias, lack of industry benchmarks, etc.). The reported 56% could be lower in reality. Why is that? What's holding L&D back from measuring business impact?
Why L&D Can't Measure Business Impact?
According to the findings, there are three significant barriers for those who do not measure impact:
- Accessing a clean, consolidated view of learning ecosystem data is hard.
- L&D faces challenges around securing stakeholder buy-in.
- Correlation vs. causation—how do you confidently attribute impact to L&D efforts?
Barrier number 3 stands out for me. Every time I see an em dash sticking to the words in the middle of a sentence I have a feeling it's hiding something. But seriously, we need to start with fundamentals first:
How do you define business impact? What's L&D's role in the business?
If your definition of business impact is bottom-line profit and L&D's role is to provide training content that confidently drives that change, you may never find peace. Isolating the direct impact of a single training program as the sole driver of business impact is pretty rare. That does not mean the strategy is not to measure anything, though.
Speaking of L&D impact measurement strategy: if you stopped reading this right now and called each colleague of yours asking to define your organization's learning strategy, would you get the same, clear, concise response? In other words, are you all on the same page? If not, you might want to calibrate first before fixing things.
What Can You Do Now To Improve Your L&D Impact Measurement Strategy?
The survey highlights key factors that can help define and improve the current L&D measurement strategy. Overall, I suggest thinking of the strategy as a journey, not a quick fix. You don't need to perfect, but you do need to make progress.
Strengthen Business Alignment
Engage with business leaders early in the program design process to define success metrics that align with organizational goals. Ensure that all learning initiatives are tied to Key Performance Indicators (KPIs) that matter to the business. Prioritize resources against key initiatives to avoid busy but not impactful work.
Advocate For Resources
Speaking of resources...make a compelling case for dedicated budget and analytics resources by demonstrating the potential ROI of effective learning impact measurement. Use data from pilot projects to show how targeted investments can lead to significant business improvements. This may be a "chicken-or-egg" dilemma: how do you show potential business impact without documented business impact? Start small with a pilot. Select a business leader who is interested in getting actionable data. Document and share the case study.
Leverage Technology
Invest in or advocate for technology that can integrate and standardize data across the learning ecosystem. This will enable more sophisticated analysis and make it easier to demonstrate the impact of learning initiatives on business outcomes. I've seen two approaches to this problem: a) building a cloud-based data-lake that connects your traditional SCORM-type data with HR and performance data in a single solution for analytics; or b) implementing a more future-ready, xAPI-based (experience application programming interface) solution such as the Total Learning Architecture.
Don't think of technology as the fix for your legacy problems. Sometimes, letting go of the legacy approach opens up new possibilities. It is like reframing the question. Maybe it's not about how to use gen AI to create courses faster, but rather how to use gen AI not to create as many courses.
Upskill The L&D Team
Sometimes, hiring dedicated analytics staff is not feasible. No matter how well your team is aligned with the business and what technology is ready to support, if your team lacks the skills of speaking data, the language of impact, you will have issues with your L&D impact measurement strategy.
Focus on upskilling current L&D team members in data literacy, data analysis, and fundamentals of AI. Foster a culture of curiosity where team members are encouraged to explore data and derive insights that can inform decision-making. One of the best ways to get buy-in from business stakeholders for resources, tools, and technology is to provide them with actionable data they can use to make proactive decisions, rather than just documenting past accomplishments.
Prioritize High-Impact Initiatives
"We are too busy!": this is often a symptom of a prioritization problem. I've spent over 25 years in the corporate world, delivering thousands of projects. And I can tell you with 100% accuracy how many times a stakeholder came to me and said: "I need this learning solution but don't worry about it, it is not a priority," and that is none. It is a priority, always important, always urgent. Therefore, you need a prioritization strategy, process, and leadership support to implement that efficiently.
Focus measurement efforts on high-stakes projects first. Compliance training or large-scale initiatives with significant business implications are often nonnegotiable. Next, prioritize those projects where the stakeholders are willing to work with you on collecting data and measuring impact.
"We can't say no!", you might say - and you're right. If you just say "no" to projects without a consistent prioritization strategy, the business would just go around you. This is a conversation that needs to happen ahead of time. Be a consultant. "Walk the floor" to understand the business problems before they come to you asking for training (especially when training is not the answer). You may also create a self-service tool kit. If the business wants to create their own solutions when you do not have the resources, give them the tools and guidance on how to do so.
Communicate Success Stories
Another lesson I learned in my career is that it is not the data that convinces the business to invest in learning. Data is not enough. Even if you're right, it is often not enough. Why? Because our goal is not just to be right, it is to make a difference. And you make a difference through people and change. Changing people's minds is not about the amount of data you present. It is the language you speak. How you tell the story matters as much as what story you tell.
Build a case study and show the results. Don't try to convince other business leaders about the what and how details first. Show the result of a project to demonstrate the why. Reframe their mindset from "What is this?" to "How did you do this? How could we do something like that?"
Share the results of impact measurements with key stakeholders regularly. Use these success stories to build credibility and support for expanding measurement efforts across the organization. Branding, while it may sound silly, can help you focus on the message both for your organization and other leaders who advocate for you when you're not in the room.
Welcome to the journey of learning to speak data, the language of impact!
References:
[1] New Report Reveals Key Strategies for Measuring the Business Impact of Corporate L&D
