subject: How to Evaluate Training [print this page] How to Evaluate Training How to Evaluate Training
Evaluating Training is the last part of Kolb's Training Cycle and an area that is often forgotten or carried out half heartedly by Trainers and organisations. This is often due to a lack of understanding of its importance or how to carryout effective evaluation.
Evaluation of training events is important on two fronts.
For the Trainer it informs them if what they were training was received and it tells them if the event is pitched at the right level. Without evaluating you can deliver the same course year on year without ever knowing if the course is right. It is the only feedback Trainers receive on a day to day basis as to how they are performing.
For the organisation is it vital to know how the training was received to understand if it was worthwhile and justifies the expense. Having a Training policy and carrying out training is brilliant in any organisation but, if you are not measuring how well it is received or what impact it is having back in the workplace then you may as well not have any training at all. For training to be effective in the workplace it must be
i. Relevant
ii. Cost effective
iii. Adding value
Evaluating Training
There are a number of Evaluation Models out there' but, the most widely used and accepted one is Donald Kirkpatrick's original 1959 4 level model. Now there are 5, 6, 7, 8 and 9 level evaluation models out there but, as most organisation fail to evaluate on more than two levels I feel that these are redundant. If you can say without wavering or a shadow of a doubt that you evaluate at the four levels then by all means take a look at the other models. If you do not evaluate your training events at four level then don't waste your time looking at the 5, 6, 7, 8 and 9 level models why confuse the issue and have more tools that you won't use?
Kirkpatrick's original 4 level model is more than adequate to evaluate training in organisations. This is because it provides the information you need without overcomplicating the issue. The levels are
Reaction reaction of the delegates to the training
Knowledge the knowledge that has been transferred during the training
Transfer are they using or able to use the knowledge and skills back in the work environment
Effect on Organisation how has it impacted the organisation?
Looks simple and it is if you don't over complicate it.
Reaction
This is carried during the training event. It is to gauge if the delegates
a) Understand what you saying
b) Are comfortable with what you are saying
c) Understand why they doing an activity
d) Need a break
These things are gauged from asking the right questions and observing body language and behaviour. A lot of training will be uncomfortable as it will be pushing the comfort zone of the delegates further. If it is a change to a process this will be uncomfortable and you have to sell the change well in order for it to be accepted.
The other way that Reaction is measured is at the end of the Training event with Happy Sheets'. These measure anything from the size of the room to the comfort of the chairs and the variety of food on offer. These items are either marked on a scale of 1 5 (1 being poor and 5 being excellent) or with smiley faces ( J K L ) to indicate how the delegate felt about each item.
Knowledge
This level is the most important for Trainers as it provides you with information about the course you are delivering (Especially if you have designed and written it) and your delivery of it. This is because it tells you if the knowledge has been transferred and this helps you understand how good or not so good you are at explaining things.
There are a number of ways to measure knowledge transfer. The easiest being a questionnaire. To do this effectively you need to hand out the questionnaire before you start the training and again at the end of the training. This gives you a pre and post training score. There is one major rule about evaluating knowledge. That is
It is an evaluation of the trainer NOT the learners!
This is very important to remember because if the learners have not understood something or gained the knowledge you wanted them to then it is your fault. Your delivery, the methods you used, the level of language used, the level of the course etc. It is your responsibility as a trainer to ensure that EVERYONE learns on your training events.
You can measure knowledge through a demonstration of a skill with an observation sheet where you mark against set criteria.
Transfer to the Work Environment
This is one of the easiest levels to measure. If your training has come from a Training Needs Analysis[1] then you should still have tools that you used to indentify what training is needed and you can use them here as well. The easiest way to measure transfer is through observation. If training is to be effective it must be something that can be utilised in the workplace. If it is not being utilised then why did you train it? This puts no value on your training and the motives for doing it will be questioned.
Effect on Organisation
The effect of training on an organisation is a difficult to prove as other departments will always say that is was them that improved. If you train the Sales Team and sales improve the Sales Team will take the credit for doing a good job and for understanding its customers better. This area of evaluation requires checking of figures against figures prior to the training and seeing if there has been an affect.
With ROI (Return on Investment) being a new buzz word it can be incorporated in the fourth level. There are many discussions on how you measure the Return on Investment in Training and there does not appear to be a consensus of opinion on how it is done to measure it effectively.
Who is Involved
Evaluation can be done by several people although not at the same time. Level 1 (reaction) and Level 2 (knowledge) are done by the trainer or the person delivering the learning in the classroom or through whatever medium you are using. Reaction is measured throughout the training and at the end with the happy sheets. Knowledge is measured at the start and at the end. The pre-course knowledge should be measured after the objectives have been stated e.g.
Start Introduction of self and course
Domestics (Fire Exits, Toilets etc.)
Ice Breaker
Aim and Objectives
Level 2 Evaluation
The purpose needs to be explained to the delegates. Couple of rules here; don't call it a test or evaluation!!
When we get to level 3 (transfer to work environment) the Trainer and the Line manager or the person that requested the training can carry out this evaluation.
At level 4 (effect on organisation) this really falls to various departments;
1) Finance
2) Human Resources
3) Quality Assurance
4) Health and Safety
Or any other department that calculates figures to show how productive or otherwise the organisation is being.
Which levels to carry out
In an ideal world all four levels of Kirkpatrick's model will be carried out and to their fullest. We don't live in an ideal world so if you are going to measure how effective your training event was you need to do at least three levels and the first two are a must. Each level on their own are meaningless especially level 1. Many companies rely on this level to buy in the same training organisation or to see if they need a training department and it does not give the full picture. If you only measure induction training on the happy sheets you are going to think that it is the best induction package around because new entrants want to please and believe that they are still under interview conditions on their induction. If you only measure Health and Safety updates on the happy sheet you will get the impression that it is the worst training that you offer and run the risk of rewriting or changing it without knowing what to change.
Use level 1 in conjunction with level 2 and take the knowledge factor to measure the training more effectively. This way you will get a better picture of how the training was received. If you are using a questionnaire ensure that it is not too easy or too difficult. It should make the delegate think. If your learning gain is high and the happy sheets were not so good then the training was not affected by the room, the food or anything else on the happy sheet. If the learning gain was low and the happy sheets indicated there was a problem with the room, chairs, lighting etc then the environment has had an impact on the training.
With these two levels you know that the training is doing its job. The next level you choose will determine what affect it is having in the work place and it is up to you to decide which to use. Level 3 would, for me, be the next logical step and from there you can determine if you should continue to deliver the course again.
[1] The process of identifying and determining the Training Need in Organisations and individuals by using interviews, observation, questionnaires, customer complaints and various other tools that identify that there is a training requirement.
