Where do you begin with evaluation?
A few weeks ago, we ran a Learning Loop showcase event called “Taking the Fear out of ROI”. There was a great mix of people from many different organisations and to say the discussions were lively, would be an understatement!
Steven Covey, a man of many wise words said “Begin with the end in mind”. So the answer to the question in the title, would of course echo Stephen Covey’s sentiments. If evaluation has been an afterthought to the process of delivering learning, then, quite frankly it will be a waste of time. It would be a little like starting to knit a jumper, without a pattern or any thought to shape, size or colour and then expecting it to fit your needs.
Here is the essence, for a good evaluation, you need:
- A solid needs analysis, which identifies the impact you would like the learning to have on the organisation
- Stakeholders engaged at the beginning, providing you with not only the resources to identify needs, but resources and support for the evaluation. (*for more on stakeholder management click here)
- Clear organisational outcomes, which the stakeholders will monitor and measure
- Learning outcomes that support the organisational ones
- Time before the next new project to complete the evaluation analysis and reporting
- Realistic expectations from the stakeholders about the expected outcomes
There are of course other factors, but this is brief run through of the key components. The last one is an interesting one, especially when there are multiple factors which may influence the outcomes. Let us take a simple example:
At the same time as the a customer service learning programme being rolled out, a new customer management system is also installed.
In this instance the relevant stakeholders may either:
- Join forces and measure the overall impact of both
- Agree percentages of the impact of the two separately
Whichever approach is used, there needs to be realistic expectations from the stakeholders as to some of the other factors which may prevent the objectives being achieved:
- Lack of line manager support for the learners (one of the biggest reasons for learning not imbedding)
- A long enough lead time, between the learning and the measurement, to allow the learning to imbed and for results to be observed
- Time and space in the learners roles for the learning to be put into action
Again, this is not an exhaustive list, but some of the key areas that may be investigated, if the learning does not meet expectations.