Measuring learning: Kirkpatrick’s four levels, without invented numbers
Measuring is only useful when it answers a question. The well-known Kirkpatrick model distinguishes four levels: reaction, learning, behaviour and results. A learning journey gives good signals for the first two, and can help with the last two, but those also need signals from the work itself.
- The four levels
- What a learning journey gives you
- Measuring behaviour: signals from the work
- Why we do not quote invented numbers
- Does it fit alongside your LMS?
- A simple measurement plan
- Being honest about cause and effect
- Good questions for an evaluation
- What a good report for the organisation contains
- How to set up the measurement
The four levels
| Level | Question |
|---|---|
| 1. Reaction | How did the participants experience it? Was it relevant? |
| 2. Learning | What do they now know or can do that they did not before? |
| 3. Behaviour | Do they do it differently at work? |
| 4. Results | Does it deliver something for the organisation? |
The levels build on each other, but you do not have to measure them all equally heavily. Choose per learning journey which level drives the decision for you.
What a learning journey gives you
For levels 1 and 2 CoVince gives a lot. Questions get a weight and a theme, so you see scores per theme. Open answers are assessed by AI, there are surveys and question flows, and for each learner the AI writes a personal report (multilingual, as Word or PDF). You have statistics per learning journey and per learner, with clickable figures.
That tells you where a group stands, how someone thinks and where the learning journey itself is weak. These are the best signals for improving.
Measuring behaviour: signals from the work
You do not see behaviour at work in a learning journey. It needs signals from the work itself: less rework, faster handling, better ratings of a conversation, a safety report on time more often. Which signals you choose depends on the learning goal.
A practical start: choose one measurable behaviour in advance and agree who keeps track of it. If the data is already in a system, you can show it next to the learning results in a Control Room. See learning in the flow of work.
Why we do not quote invented numbers
Vendors like to promise "up to 70% better retention" or "3x faster". Such figures rarely come from your situation. They are an average of something else, often measure something else or were never researched. Whoever copies them into a business case builds on sand.
Better: measure in your own situation, compare with an earlier round or with a control group, and be honest about what you do not know. A cautious result with your own basis is worth more than a big claim without one.
Does it fit alongside your LMS?
If you already have an LMS, the measuring does not have to move. CoVince has a SCORM player and export, xAPI and LTI 1.3 with grade passback, so a learning journey also runs in Moodle, Canvas or Brightspace. You can import a Moodle backup.
A simple measurement plan
A measurement plan does not have to be big. Write down four lines per learning journey, one per level. Reaction: what do we want to know, and how do we ask (a scale question and an open question afterwards)? Learning: which themes do we test and with which questions? Behaviour: which behaviour at work is the sign of success, and who observes it? Results: which outcome for the organisation goes with it, and which measure do we already use?
Also agree when you look back: right after completion, after a month and after a quarter. That prevents measuring from staying something you will do "some day".
Being honest about cause and effect
When results improve after a learning journey, it is tempting to give the learning journey all the credit. But more happens in an organisation than a training: new colleagues, a different process, a quieter quarter. Be careful with cause and effect. Where you can, compare with a group that has not yet done the learning journey, or with an earlier period, and state the limitations.
That is not a weakness in your reporting but the opposite: whoever is honest about what they do not know is more readily believed about what they do know. See also designing the complete learning journey.
Good questions for an evaluation
An evaluation is only as good as its questions. Avoid "was the training good?", which yields politeness. Ask what is usable instead: what will you do differently this week? Which part was least clear? Which situation from your work was missing? Use a scale of three or five points for the feeling and an open question for the reason.
Ask again after a few weeks, shorter: did you apply it, and what happened? That gives you a first indication of behaviour without heavy measuring. Combine it with what a manager sees, and discuss differences.
What a good report for the organisation contains
A report for managers or leadership does not have to be long. A usable set-up has four parts: what was the goal, who took part, what did we see (scores per theme, reactions, a few quotes) and what do we do now. State what you do not know, and give no invented numbers. End with a proposal: adjust, expand or stop. That makes measuring a basis for a decision.
How to set up the measurement
- Write the learning goal as behaviour and choose one level to steer on.
- Give questions a weight and a theme that fit the goal.
- Have the learner hand in something you can assess afterwards (open answer, conversation).
- After the first group, look at the reports and adjust the learning journey.
- Agree how you follow behaviour at work and when you look back.
The outcome is not a number but a better learning journey, every round. See designing the complete learning journey.
Frequently asked questions
What is Kirkpatrick’s model?
A model with four levels for evaluating training: reaction, learning, behaviour and results.
Can CoVince measure behaviour at work?
The learning journey mainly measures reaction and learning. Behaviour needs signals from the work itself, which you can show next to the learning results.
Do I get a report per learner?
Yes. The AI writes a personal report per learner, multilingual, as Word or PDF.
Does it work with my LMS?
Yes. There is SCORM, xAPI and LTI 1.3 with grade passback.