5 General Rules for Workplace Tests Part-1
RULE #1
Do Not Trick Them
If you have not taught “it” in the training, it should not be on the test. In addition, your test questions should be stated in the same manner they were stated/taught in the class. For example: if you teach the three characteristics of steel, don't ask: Which one of these is NOT a characteristic of steel. It's hard for most people to have success with "null" answers and more importantly, why reinforce what you don’t want them to remember?
Interview with Learning Expert - Will Thalheimer
Will Thalheimer is a learning professional and researcher whom The Training Doctor recently interviewed
T/D: Let’s focus on learning design and the research that you've done in this area.
Thalheimer: One of the things that I think a lot of us forget is that we're not really trying to create learning as much as we're trying to help our learners remember what they've learned. We’re trying to make sure that not only do they learn it, but they're able to retrieve it from memory at a later time. We not only want to increase learning, making sure they understand, but we also want to minimize forgetting. Minimization of forgetting is something a lot of us don’t think about.
When I looked at the research there are three things that are very powerful in allowing people to minimize forgetting. I'm going to go through the list. The first term is 'retrieval practice'. That means giving learners practice retrieving the information from memory like they would in the real world. Too often we present information that helps them learn, but that doesn't help them practice retrieval.
T/D: We assume that they'll remember it at the right time. That’s a good point.
Thalheimer: If people go down the forgetting curve and they can't remember it, which often happens after learning, the training was a waste of time. One of the things is retrieval practice. The second thing is 'context alignment'. People have to perform in a context - in a real situation. In the environment there's visual cues, noise cues, there's smell, there’s all kinds of things.
T/D: Things that brought you to this point - something that led you to now make a decision, right?
Thalheimer: Absolutely, all these environmental cues trigger memory retrieval, bringing information into working memory. One of the things that researchers have found is by aligning the learning context with the performance context. You have some of the same cues in the learning context that are in the performance context. When people go back to the performance context they are more likely to remember what they learned.
T/D: Can you give us an example?
Thalheimer: Absolutely. It's interesting, the army knows about this and pilots and people in dangerous situations - we need to scaffold them up to the place where they can deal with the stress of their jobs AND still make decisions. They may need to develop an understanding of things in a non-stressful situation, but in order for them to be able to retrieve the information in that real world stressful environment, we can maximize that retrieval if we put them in that kind of stressful environment.
T/D: What does scaffolding mean?
Thalheimer: Well, scaffolding means start slow and progress to more difficult.
T/D: Until it's as realistic as possible.
Thalheimer: Correct, as realistic as possible. That's why the army has people with real bullets flying over their heads because that's a real world situation
We've gone through retrieval practice and context alignment. The third thing is 'spacing' or the spacing effect. It’s one of the most studied phenomenons in all research psychology. It's the notion that if you space repetitions over time they're much more powerful than if you space them over less time. Those three things enable people to remember over a long period of time. They minimize that forgetting curve.
T/D: The fact that we send people to one training class and then send them back to work is fruitless.
Thalheimer: Fruitless may be too strong, but it's certainly not the best design. One of the things that people need to be able to do as a trainer is not only help people understand and learn, but also minimize forgetting. That's what we're all about, helping them remember on the job.
= = = = = = = = = = = = = = =
Will Thalheimer founded Work-Learning Research which is a consulting practice that helps clients build more effective learning interventions. He’s been in the learning and performance field since 1985. His professional focus is on bridging the gap between the research side and the practice side. Visit his excellent blog here: http://www.willatworklearning.com/
The Key to Outstanding Learning Outcomes: Design Learning Processes
Designing training programs can be an arduous process if you take on the responsibility for designing all the content yourself. You can lighten your load and also achieve a much greater learning outcome by designing learning processes rather than learning content.
In this article we will share three ways to design learning processes.
1. Use real work.
Learners prefer to accomplish real work while they are in the learning process. So rather than create a contrived situation or a case study scenario that is “representative” of real life, instead have the learners work on real work tasks. For instance, a financial services firm wanted to teach its sales people to read financial statements in order to find cross-selling opportunities in their current client-base. Rather than teach the sales people how to read “generic” financial statements and then leave them to transfer that knowledge and skill to their own client's financial statements, the learning session required them to bring the annual report from two of their current clients. While they learned to read a financial statement (such as a cash flow statement or a profit and loss statement) the learners were working and reviewing the actual financial statements of their own clients. This not only resulted in a better understanding of the learning but it also resulted in the learners being able to have actionable findings be the end of the training class.
2. Create the learning in real time.
Rather than teach your learners a new concept or skill in a large block of time and then expect them to be able to transfer all of that information to their real work responsibilities, break the training up into smaller, actionable learning objectives and on-the-job tasks. That will allow them to implement their new knowledge and skills in smaller chunks and result in more successful implementation on the job.
For example, a sales organization was training their sales people to listen for cues from prospects to better gauge if they could ask for an appointment or not. After a period of time in the classroom, in which the sales people/learners learned the 5 types of responses which would either open a door for them or not, they were then given an hour to return to their desks and make up to 10 phone calls from their personal prospect list; making notes about types of conversations they had and whether they were able to secure an appointment or not. This approach not only allowed the salespeople to accomplish some real work during the training process, but also created a rich discussion upon returning to the training room because they each had real-world experience implementing their new skills and were able to compare and contrast their outcomes and ask additional questions of the instructor.
If they had simply completed the training and then returned to the job to (attempt to) implement what they had learned, when would they have ever had those rich discussions that serve to cement the learning for adults?
3. Have the learners contribute the content.
A third sales organization wanted to teach overcoming objections in relation to a very complex product that their sales people sold. Rather than try to anticipate all the objections and give the sales people pat answers in reply, the training was designed to first solicit the “5 toughest objection you've encountered when attempting to sell xyz” and then a game was created, dividing the larger group into 3 teams and giving each team the opportunity to craft an appropriate rebuttal to the objection.
The learning process went like this
Team A stated an objection to Team B~Team B had a period of time to craft an appropriate response~Team C had the opportunity to challenge Team B's response~Team A chose what they thought to be the best response and awarded a point to either Team B or Team C accordingly.
This learning process continued in a round-robin style, with Team B next sharing an objection, Team C getting the first opportunity to reply, and Team A being given the opportunity to “challenge” team C’s response, until all of the 15 toughest objections in regard to that topic had been addressed.
This process allowed the learners to share their real-world problems and to get the best and the brightest to assist them with being better prepared the next time they heard that objection.
The next time you are attempting to design learning content take a step back and see if instead you are able to design a learning process that better assists the learners in working with and assimilating that content.
Learning processes can often lead to greater learning outcomes because the learners are more engaged with the content, identify with it more clearly, and have less trouble transferring what they learned in the classroom to what they do on the job. An added bonus, from a logistical standpoint, is that designing a learning process requires much less updating in the future should the content itself change.
Adobe Connect and Your Virtual Learning Offerings
Do you (or do you plan to) deliver classes using Adobe Connect? Would you like to utilize ALL of the features of Connect in a way that enables you to have engaging learning events?The BEST of instructional design for the virtual classroom AND technology tips combined in one class.
One recent participant had this to say:
I have met people who understand learning and I have met people who know Adobe Connect, but I have never met a person who has both those skills at once. This class was outstanding.
Instructional Designer, Navy Federal Credit Union
Offered by Training Magazine and presented by The Training Doctor.
Sign up before March 29 and save $150. Group rates available. Get all the details here.
Use your Network to Create + Supply Employee Training
Too often we isolate ourselves and think that we are the only organization that has a particular problem; for instance, quality control issues or the need for sales training on a particular piece of equipment that your company resells. We put a lot of time and effort in to creating a “customized” training program when, in all likelihood, that training program has already been created by somebody else.
Suppliers
Many suppliers already have training programs in place to train their own workforce. Simply ask if you can borrow their programs or perhaps pay a fee to use them with your employees. Often, it is possible to have your supplier-company salesperson or another representative provide the training for you. Case in point: a retail organization wanted to train its store clerks in athletic footwear construction and the best use of each brand or style of footwear so that they could be more responsive to customer needs. The first thing the operations manager of the retailer did was call all eight salespeople at the athletic footwear, manufacturers, and ask what resources were available that the retailer could have or borrow. All eight footwear manufacturers were eager to provide information and assistance with the training project. In fact, all eight salespeople came in on the same day, in one-hour blocks, to present what they felt the clerks should know about their particular brand of footwear and how it was different from the other manufacturers.
Another example comes from a restaurant trying to increase its dining checks by increasing the level of liquor sales per party. The focus was on wines and how they complemented the various menu items. One liquor salesman, in particular, had a passion for wine and was thrilled to be asked to provide a 90-minute workshop for the restaurant's employees on the different types of wines and the different menu items that they complemented. The salesperson did the training on his own time, certainly with the expectation that his sales would increase if the restaurant’s retailing of wine increased, but also because he was thrilled to talk about something he really loved.
So first: think about how your suppliers can provide you with training or training resources.
Client Companies
Another approach is to look to your client-companies – perhaps your clients have already solved the challenge of lean manufacturing in their organization and would be happy to include your employees in the training that they already offer, or to loan their trainers to your organization to conduct the same type of training. Don’t think of yourself as isolated and trying to solve a problem that is unique to your organization because 80% of the time it is not unique – pick up the phone and call a few of your clients to see if they have faced (and surmounted) the same challenges.
Cooperative Relationships
The last suggestion is to actively seek out similar companies in your industry. As an example, a southeastern state found itself with under-qualified bridge and road inspectors due to an early retirement package offered by the state. The employees that were left did not have as many years experience on the job and no longer had mentors from whom to learn. The state realized it had to create a new training process, but was wise enough to realize it could not be the only state in the nation with that need. So the state’s transportation commissioner called 30 of his colleagues across the US and asked them if they had encountered the same challenge. Of the 30 colleagues contacted, two offered their training program (materials) in their entirety to use as a model or to use carte blanche.
Before attempting to solve a “training problem” in your organization think creatively and cooperatively about who else might also have the need and has perhaps already solved it for you. Very often you will find that you have resources and support at your disposal, quite in abundance, simply for the asking.
Interview with an Expert: Susan Boyd
How to Make Learning Stick in Online Courses
T/D: Susan, why is there the perception that e-learning doesn't stick and what can we do about it?
Boyd: I think there are a variety of challenges that can cause an e-learning program to fail. One in particular would be technology barriers which result in the trainee/learner not having the proper access to get to the course. Often a trainee/learner gets to the course only to find the content is really not pertinent to their job needs or find they have to go through 100 page turning screens just to find a small nugget of content that is pertinent. It’s quite possible the training will not stick because it's almost as though we're hiding the forest with the trees. Trainees are not getting the overall picture.
Some trainees/learners really just want to get through the content thinking the ultimate purpose to e-learning is to get to the end. Many are only looking to find the “next” button and continue to push it until it’s at the end of the session.
T/D: What are some tips that can make this better? Are your tips coming from the perspective of instructional designers who are the ones who create the training or are there tips for us as learners?
Boyd: Most of my tips have to do with content - I think that's the instructor and designer. How to break up the content into chunks of learning, how to make it as job specific as possible. Try to give the learner control of the content, make it interactive - those are things that instructional designers can do.
Preparing the learners and preparing the managers are also really key. Many managers thought e-learning was just this magic button people push on their computer and, through osmosis, people would instantly learn the topic. Managers don’t often invest enough time in following through: Did people do it? Do they understand it? And are they applying it?
The key thing for the learners is to ensure, they can access it. We have a generation of learners today that are more internet savvy - so that's wonderful. But their computers at home are typically better than their computers in the office.
We need to make sure we're preparing them by whatever wizards they need to test their system. Whether it's a webinar they're going to attend or it's how to get to the learning management system to enroll in the online course. It needs to be made as easy as possible. Maybe some frequently asked questions, or getting started guides are helpful.
T/D: That’s not as much making the learning stick as making the environment more accessible for the learning to stick.
Boyd: Right. Learning can't stick if you can't get to the course. Good point. I look at this as a multi-faceted problem. Part of it is the content design with the instructional design; part is working with the learners; part is working with the managers - providing them the right support material for what they need to do.
In terms of making learning stick, it’s important to make it real world. People can relate to it and find the information in the course, but can also find it outside the course. If there's a reference card or frequently asked question which can only be found in the course, that becomes the challenge because these are the types of things people need to access outside the course. So there ought to be a way that they know where to find those same types of online references.
T/D: What tips do we have for training designers then?
Boyd: I think it's engaging and having the learning relate to something - hopefully it's a real world example. Making some connection ia key part of it. Using graphics and pictures - for example in safety courses - I worked with one client and the pictures we use are pictures of people on the factory line. This particular client is a manufacturing company and there are examples of them where something is unsafe. Maybe they're not wearing the proper protective equipment or there's a tripping hazard. You're taking these real pictures and saying, what's wrong?
T/D: That's fun!
Boyd: Or saying, there are six things wrong with this picture, can you define them or click them. In that case defining them sometimes is actually click in the area where you think there's a problem. So it is forcing the learner to do something – we want the learners to do more than click on the “next” button.
TD: What I hear you saying, is in order to make e-learning stick, you have to understand how the learner is going to use the content and support them through the process. It's about managing the manager's expectations on how the learning will be used or what the end result of the learning is. Then designing it so that it's mentally stimulating and engaging for the participants, which is a design process. It's pulling the learning from the participants which is what we normally do in the classroom, but we don't think it can be done online in my estimation.
Boyd: That's the challenge. E-learning was supposed to be able to get the right content to the right person at the right time. They were equations that we had trouble with in the classroom - we could never deal with the right place and time - e-learning kind of takes away that barrier. But we're still struggling with the content.
The design of the content has to understand the environment in which the content is going to be used.
T/D: Thank you Susan, The most powerful thing you said to me today was that when we take the learners out of the equation, we design bad e-learning. That's really powerful because I think too often we think of e-learning as the 'e' part and we have to start thinking about it as the 'learning' part.
= = = = = =
Susan Boyd has over 30 years' experience in training and computer education, with extensive experience in planning and managing computer training projects. She has directed and participated in all phases of the projects including needs analysis, course development, train the trainers, course delivery and follow-up. Susan has developed and delivered training in a variety of formats including instructor-led classroom, virtual training sessions using a webinar, and online asynchronous e-learning courses. You can learn more about Susan and contact her at www.susan-boyd.com
Moving your classroom content to Online?
You may be interested in this 4-class series, offered by Training Magazine and presented by The Training Doctor.
Complete details here: http://www.trainingliveandonline.com/vILT_Design_Adobe_Connect.cfm
Save $150 if you register by March 29!
Creating Tests in Your Workplace?
See The Training Doctor later today (2:00p EST start) at Training 2013 to learn how to design test questions for the workplace.
Do it right from the start, or suffer potential consequences.
Getting training to your employees, a challenge?
In a poll of learning professionals conducted by ON24 earlier this year, respondents said their biggest training challenge is training an increasingly dispersed workforce (33.3%).
The Most Important Key to Facilitating in the Virtual Classroom
We all know good facilitation skills: ask don't tell; encourage discussion among participants-not just between you and the participants; play devil's advocate to challenge people's thinking, etc., but when we move from the classroom to the virtual classroom we seem to revert immediately to lecture and presentation mode.
If you think straightforward lecture is a snoozer in the classroom, it’s ten times worse in the virtual classroom when the only thing participants have to keep them engaged is the sound of your voice. In order to deliver online classes that are engaging, facilitation skills are more critical than ever.
A common complaint of new online facilitators is that the participants aren't paying attention and are frequently multitasking. That is only true if you are not facilitating. If you are facilitating, your participants would be too involved with the learning process to have the time or the inclination to multitask.
According to Wikipedia, a facilitator is someone who helps a group of people understand their common objectives and assists them in planning to achieve them. Another definition, also from Wikipedia, states that a facilitator is a person who makes it easier for other people to accomplish their objectives by offering advice and assistance. A common theme here is that the facilitator is not the focus of the learning event, the facilitator brings the learning about through the training participants.
The best virtual classroom learning experiences put the dialogue and the learning process firmly in the hands of the participants. If you're bringing people together online in order to simply make a presentation - don't. Record the presentation and let your participants view it on-demand. Bring people together online for the kind of interactive and collaborative outcomes that can only be achieved when we bring people together.
ISD for the Sync Environment - learn it here!
Wondering how to move your classroom-based classes to the live online environment? Join us at Training 2012 in Atlanta later this month for the quick and low-down on ISD for the Synch Environment.
Training Triage
One of the things The Training Doctor specializes in is Training Triage -that is- why doesn't training work?Throughout our 20+ years in business, we've discovered the answers to this question.
This month, we share with you one of the reasons: SME designed training.
SMEs (subject matter experts) are a fount of information. We couldn't do our job without them. Which, if you rephrase that, would be better stated: We should do our job with them (and not have them do our job).
Too often we are called by an organization which has training in place but it is not achieving the results they wanted and they cannot figure out why. Here are some recent examples:
Project management training
New hire insurance sales training
Field service representative (computers) training
Medical imaging technician training
The list goes on and on. Inevitably, upon reviewing the current training materials and process, we say to our client / prospect: The content is good. It's all in there. It's just not designed in a way that anybody could learn from it.
Information does not equal knowledge. It most certainly does not equal skill or ability. Yet, most subject-matter-expert-designed training is a fire-hose of knowledge, delivered (more often than not) via lecture. One client DID provide a very helpful 70-page, single spaced, typed "learner guide," as well. The learners not only didn't get it in class, they couldn't find it after class either.
Interestingly, as we were deciding upon what topic to feature this month in Training Triage, we came upon this blog posting by Clark Quinn. In it , he laments about a project he is working on which is based on a "design" from an SME. He labels it, "too-rote, too-knowledge dump." Two of Quinn's comments are both humorous and insightful:* SME's don't know how they do what they do*
Learning design isn't for the wimpy.
Casino revenue earmarked for workforce training
Beginning in January, manufacturing businesses in the state of Ohio are able to apply for reimbursement for incumbent worker training. This funding is linked to casino revenue / profits which began to accumulate in 2011. This is unique in a number of ways:
The state government is taking revenue from one industry and applying it to another
Manufacturing is the only targeted industry to receive this benefit; organizations must be in advanced manufacturing, aerospace, autos, energy, food processing, IT, and polymers and chemicals.
The reimbursement is specifically earmarked for incumbent workers (whereas must government sponsored or supported workforce training is to get people jobs / new-hire training)
The companies can only receive reimbursement once people have completed the training
Part-time workers are eligible (a minimum of 25 hours per week)
The employer must pay at least 150% of minimum wage to the eligible employees
The training can be provided by a trade school, trade association or other 3rd party entity and certificate completion is allowed, up to $4000
The state government is only reimbursing up to 50% of the cost of the training
A state official has said that the underlying motivation is for Ohio-based manufacturers to remain competitive by investing the technology and their workforce skills in new(er) methods.
This seems like a very well thought out plan for employee development, sponsored by a government organization. Bravo!