How to Conduct a Level 3 Evaluation

According to best-selling author Marcus Buckingham, performance ratings rely on "bad data." Labeled the "idiosyncratic rater effect," he states that who we pay,  what we pay, who we promote and the training we offer is based on the assumption that one's "rating" is reflective of the one being rated - when in fact it is reflective of the one doing the rating.

Often, when conducting Level 3 evaluations, we ask a manager or some other entity to "rate" a newly trained employee in order to confirm they have learned and can apply their new skills on the job. In order to not succumb to the idiosyncratic rater effect, it is wise to use an impartial observation sheet, so that the rater simply confirms whether or not the employee is performing the job as expected.

For example:

Comments

Answers phone within 3 rings ¨  Yes ¨  No

States name and badge number ¨  Yes ¨  No

Asks permission to put caller on hold ¨  Yes ¨  No

But even a seemingly straightforward observation checklist can be fraught with imprecision that may skew the rating results.  Before designing a Level 3 evaluation for your own training, consider these factors which may impact your learner's reported "success."          

Who should be the observer?        

What should be the setting?          

Should the trainee be told in advance they will be observed?          

Does the time of day matter?         

Does the day of the week matter?           

Should it be a simulated scenario or a real life one?         

How long should the observation last?           

Should the observer give them feedback? When?         

Should the trainee explain what they are doing?

If you need assistance with designing training evaluations for your organization, visit our web page.