June the 11th 2019 was the date of the Learning Technologies Summer Forum in ExCel London. I was honoured to run a session on “Finding the Story in the Data” and here are some of my notes and thoughts about the session.
This session was a practical nitty gritty sort of event. I think people did forgive me if I was teaching my grandmother to suck eggs but I do hear from a lot of L&D people who just don’t know where to start. Data is all over the place and you can easily get swamped. So the purpose of this session was to get people started and get some confidence in looking at data in a practical way.
I started by asking a question: “Why bother collecting or analysing data?”.
Here are some of the reasons collected from the Learning and Skills group webinar by the same name the week before.
The chart was put together by Laura Overton and reproduced with her permission.
The two main reasons as you can see are to improve the user experience and also understand the effects or benefits. Not surprising really and in a report by Towards Maturity from 2018 they speak about 91% of the top deck saying that their learning interventions were aligned to the business goals. In order to do that, you need to be measuring what you are doing.
Other reasons may be:
- To check if things are going to plan
- Demonstrate the value brought by L&D
- Transition from learning provide to performance enhancer
- Avoid the sheep dip approach
- It is expected
- Stakeholder buy-in
My engineering brain….. in a former life I was a chemical engineer and fuel technologist and if you think that it is all about data and analysis with no room for intuition, then let me share a little story:
As an engineer, gathering data to site wind turbines, I became very skilled at finding appropriate sites just by looking at a map. This helped me to narrow down where to look from a myriad of places, that might be suitable. I would look at the maps, gather data from a mast and correlate it to the nearest met station. It is no different in L&D. You can use your intuition to see where things might be going wrong, from the data that you are already collecting and from your stakeholders. This means you can collect limited and focussed data to confirm your suspicions, to begin to find the story in the data.
Understanding the link between data and performance is crucial, as per the diagram below.
Knowing when to collect quantitative or qualitative data is also important.
Working through a case study helped participants decide when it was appropriate to gather quantitative data and then qualitative. A crucial part of this thinking was to think broader than the case study which is a great piece of advice to anyone doing their own analysis. Look and see what is happening in your industry just in case the sudden drop in sales is industry wide and not just a blip in your own organisation. It could save you a lot of time!
I then challenged the participants to say what they saw in a number of different graphs , encouraging them to be playful to find the story in the data. Sometimes the graphs raised more questions than they answered but it certainly gave everyone an insight into how easy it is to use Excel and simple charts to uncover that story.
I just had to share this picture from LTSF19 – Rachael Orchard, my fabulous host for the session, kindly brought her stormtrooper so we could endlessly make Star Wars puns and then playful Don Taylor agreed to pose with us both!
My thoughts are meandering today onto my book “How not to waste your money on Training” (work in early progress) and after posing the question above on Twitter, there were some interesting answers. The whole storify for the tweet chat can be accessed here, but there were a few answers that made me ponder more:
- Impatience from L&D and the client
- Lack of accountability for L&D
- Distracted by the new and shiny
- Not realising it is not a static process
So my next question has to be, “How do we help others in L&D see that if they get the analysis part right, then this follows?”:
- Respect and inclusion from the business
- Flexibility for the organisation
- Demonstrating value, so getting budgets is easier
- Getting to play with the shiny stuff, to enhance the learning experience
My thoughts are that there is a fear in L&D of gathering data, analysis, interpretation, challenging the norm and having the gall to ask “Do we really need this?” or “Is this really a learning gap?”, for fear we might be out of a job.
Thing is, if we don’t start asking these questions, we may be out of a job anyway…..
This is the third in a series of blogs inspired by David Hayden, at the CIPD NAP(Northern Area Partnerships) conference June 2016, in a short workshop. The title of his workshop was “Is L&D prepared for the Future of Learning?” and the basis of the discussion was around key statistics uncovered in the “Towards Maturity” report of April 2016 “Preparing for the Future of Learning”. The third question, not the statistics in the graphic, caused me to do some deep thinking!
So, let me tell you a little bit about my thinking in term of learners, digital stuff and also what my experience has been. I am an ex-engineer (if you can ever really leave that?) and a former IT trainer for IBM, so digitally, I would say I am maybe more comfortable than the minority, as keen as the majority, but not as convinced as the digital evangelists.
I have run webinars, created short learning videos, taken part in Twitter chats (LnDConnect) and learn from my own professional learning network, I blog regularly, share updates on LinkedIn and engage in forums, created online polls, used online reflective apps like Brainscape, I have designed blended learning programmes and generally embraced new technology, where it can accelerate and enhance the learning experience. Let me make it quite clear, I am fluent and practised in digital and I use it as an ingredient to a rich blend of many other methods. It is not the first or only thing I think of when looking for a learning solution. So this question is what has caused me to think deeply. “Are L&D thinking digitally?”
If I am baking a cake, I use the right tools for the job and in L&D I think exactly the same. I consider carefully*:
- Budget and resources
- Location(s) of the learners
- The topic
- Depth of the learning required (so I may layer different methods)
- Commitment of the stakeholders
- Size an culture of the organisation
*See also blog on LNA
The question“Are L&D thinking digitally?” implies that this is how we should be thinking. Digital is not the answer to every L&D problem, it is part of a toolkit available to L&D professionals to create a great blend of learning that will maximise the effectiveness of any planned learning interventions. It is very easy, with the latest, shiniest digital tools, to be thinking “Oh golly where can I use this?” (in my giddiness – I have been there!), whereas we should be thinking about:
“What will work best in this situation, with these learners and to achieve the best organisational outcomes?”
So with this in mind, I would change this question to: “Are L&D thinking digitally, in an appropriate way?“. Maybe its semantics…. what do you think?
This is the first in a series of blogs inspired by David Hayden, at the CIPD NAP(Northern Area Partnerships) conference June 2016, in a short workshop. The title of his workshop was “Is L&D prepared for the Future of Learning?” and the basis of the discussion was around key statistics uncovered in the “Towards Maturity” report of April 2016 “Preparing for the Future of Learning”. David presented around an infographic, part of which is displayed to the left. What fascinated me were some of the statistics in this segment.
In the report, the survey showed that 17% of those polled, measure business metrics to improve business performance, but 86% would like to improve business performance. That is a huge mismatch and it got me thinking “Why then, if the will is there, people do not measure L&D’s performance against business metrics?”
So I have a theory and it all stems from “Begin with the end in mind”. It is all very well to have a great intention of “improving business performance” and a whopping 86% of the respondents wanted this, but you have to start off on the right foot. At the beginning you have to do the right kind of analysis to determine the needs and the outputs, making sure that there is in fact a strong link between them. Then you need to:
- Identify those people who have a stake in improving the business
- Of those stakeholders, identify where they are on the stakeholder analysis grid, that way you know where to focus your efforts
- Be part of the business and have your finger on the pulse, so you always get the bigger picture
- Ask questions about organisational benefits and impact, not just learning outcomes
- Do a thorough needs analysis (not just an LNA or TNA) to uncover what individuals, teams and the organisation needs
- Set objectives with the stakeholders and have targets that THEY can measure success against
- Agree post learning activities and follow up
- Keep them up to date with what is going on and get them to support the learners
- Check in at various points and update them on progress
- Ask the stakeholders how the measures put in place are stacking up
I’ve written before about “Needs Analysis” and spoken about it at the CIPD L&D exhibition in May 2016 – what surprised me was the number of people that are REALLY interested in this topic! The last point in the list, is a crucial one, because another reason I believe only 17% measure business metrics in evaluation is that the best people to do the measuring are those who are most interested in those metrics and have EASY access to them!!! Is that a little too obvious?
Now L&D does not sit in some sort of vacuum or at least it shouldn’t. L&D is an important and necessary part of any successful businesses strategy – and if it isn’t we should be asking ourselves “why?” and “what can we do to become essential and not just a cost?” If you are not conducting a needs analysis that involves the correct stakeholders and using or developing metrics then what are you doing?
This is what David Hayden, L&D Professional at the CIPD said about the statistics:
“The challenge of 17% only measuring demonstrates it can be a massive challenge and rethink on what we measure. My advice is to start small and build up – pick one or two projects or interventions and work on the business metric links for those – and make it explicit in any pre intervention communication, during the intervention at regular points and in any post intervention communication. Become known for knowing the business goals!! Be that role model!”
“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”
― H. James Harrington
If you would like to discuss further with a group of L&D professionals, then come along to our next free Showcase event : “Taking the fear out of ROI”
Earlier this week we set off for London for the CIPD L&D show, to exhibit for the second time. This time seemed easier, I knew what it might be like. What I was blown away with was, the interest in my session on “The Secrets of LNA – evaluating business alignment”.
There were enough seats for 70 and standing room at least double that. As I spoke, eyes fixed on me, heads nodded and people identified with the content.
I began quoting from the CIPD L&D survey of 2015: “Of the organisations polled only 25% said that L&D are fully aligned to the organisation“. So this worries me – what is happening in the other 75%? Where does your organisation fall? In the 25% or the 75%? If you don’t know or if you are in the 75%, consider this. How would you like:
- L&D to be the change agents for your organisation?
- It to be easy to justify budgets for L&D interventions?
- When there is a downturn, L&D is not cut, but people are invested in?
By analysing the needs of your organisation before delivering any learning or training, you may find the things above become a reality!
TNA? LNA? NA?
Is this all just semantics? Are they just all the same? So here is the thing, if I conduct a Training Needs Analysis, the solutions are always going to be training. It is a little like having only a hammer in your toolbox and so everything looks like a nail. Often organisations who conduct only TNA’s may be either very technical in the learning they deliver, or it could be that they do not know much about the organisation and how it operates.
So how does a TNA differ from an LNA you might ask? So an LNA will be broader in its outlook, the equivalent of having now a hammer and maybe a wrench and a screwdriver along with some allen keys in your toolbox.. The outcome will always be a learning solution whether it is a book to read, some coaching, a webinar or a full blown qualification. What I would love to happen and here is where over the last few years I have been trying to use my Jedi mind tricks (I do know I am not Yoda btw), is when you are conducting an LNA, you ask some questions:
- Is there something happening behind the scenes that I need to know about?
- Is there something missing?
- Is something not happening?
- Is there something besides learning that these people need (eg more resources, better processes, more support etc?)
Those are just a few to get you started. These are great questions to ask if you are trying to dig deeper and look beyond the traditional training or learning needs. For this to be successful though there are some things that you will need in your personal toolkit:
- An air of curiosity
- A willingness to find out more about the organisations and how it works
- The ability to speak the language of the stakeholders and not just in L&D speak
- Persistence and courage to challenge when people just tell you to “DO it” (the training that is)
- An overview of what the culture is like and how the organisation is structured (this can be key in determining how easy it is to get people on board and change minds. For example a company with a hierarchical structure and a blame culture will resist change and pass the buck. Whereas a matrix structure and a culture of empowering, will welcome your curiosity and fresh eyes to see what might be going wrong.
- Infiltrate the organisation so you have your finger on the pulse of what is happening, now, not 6 months ago
Sometimes we may not be able to foresee when we need to do an LNA. Have a look at the picture below to see some of the instances when they can be planned and when not. Try as much as you can to plan in your LNA’s (always thinking about what might be going on under the surface). Once you start doing regular LNA’s and demonstrating the value your solutions bring, it will become easier and easier to get the resources you need to do a valuable LNA and any subsequent solutions.
Once you know you are going to do an LNA, you then need to choose some suitable methods. Below is a table of many different LNA methods. You could start by trying to sort them according to whether they are high/low cost and whether they are suitable for individuals or groups. This is one way to see which methods are going to be most suitable for your situation. You will also need to consider some other criteria, to be able to decide which methods are most suitable:
- Your budget
- Resources, such as people and tools
- Commitment from stakeholders – without this, it does make it harder*
- Size and culture of your organisation
*Read this blog about stakeholder management
So finally …. here are some of the secrets of LNA (I am sure you knew these already!)
- Know the difference between an LNA, TNA and NA (remember the Jedi mind tricks!)
- Choose the most suitable methods (use triangulation – 3 methods to get a broader picture)
- Plan the LNA when you can
- Always keep the end in mind so that you are aligned to your business
Thanks to everyone who came to the session and participated. We were truly overwhelmed by the numbers who were there and also the numbers of people who spoke to us saying “We are in that 75% and we need help!”
This topic certainly seems to have struck a chord with a lot of people and my concern is that if you are in that 75%, you get the help you need to achieve alignment with your organisation. If you need help, then please phone for a chat to see what we could do. Phone Krys on 07952 416530 or email firstname.lastname@example.org
There is a truism that training and R&D are the first places to get hit when companies want to cut costs. But how do companies get to the point of cutting these areas that are vital to future success?
When speaking to companies I often hear of the intense pressure on L&D to “deliver” and as a result, staff are under pressure to do more and more with less and less. Let’s be clear, no organisation has an unlimited training budget (but if there is one, please contact me immediately!) but I don’t think it is necessary either.
Senior executives have to see a correlation between the value that L&D brings to an organisation and business results. If they can’t see one, then that is an uncomfortable place to have to be because it may mean a skill gap in both L&D and senior management.
You might think the way to show a link is through measurement? Not necessarily. Certainly it is much easier for large organisations to measure lots of things these days using IT, yet it does not mean that you are measuring the right things. You can only measure the right things when you have a comprehensive understanding of your business and the business environment in which you operate. Once you have found the areas, which make your business successful, doing a good analysis at the start, you can then decide how best to measure performance in those areas.
When you have made those links you will need to identify the stakeholders and agree objectives. That may sound simplistic but this is a great truth for many organisational activities – well beyond L&D! Identifying stakeholders is important and gets much more difficult with large and complicated organisations. This can also become problematical in large organisations when people who were not considered in the initial consultations undermine the L&D initiatives. There are ways of minimising this but, again, it’s for further discussion.
So, L&D can deliver! And L&D can be very effective if it delivers to objectives that are linked to real business success.
You can read more about our approach to learning through “The Learning Loop approach“.