A common issue that arises late in the sales cycle when trying to close a deal to deliver sales training is "How do you measure the impact of your course?" This question is loaded with danger...and opportunity for the forward-thinking training organization. The stack of research and literature on measuring the impact of training is high and deep, but the usual approaches have significant limitations. These limits have to be addressed to have a rigorous and meaningful assessment of the effectiveness and impact of sales training.
Level 1
Post training evaluations do a good job of measuring participants reaction to the training event and often include a component asking respondents to predict applicability to their job performance. This self-reported data correlates at very low level with future business impact (r < .2) and a learners ability to recall specific information from the training (Alliger, Tannenbaum, Bennett, Traver & Shotland, 1997). Predictions about future performance and ability to retrieve information and apply it are usually overconfident (Zechmeister & Shaughnessy, 1980; Thalheimer, 2007). While post-training evaluations are a good starting point for measuring sales training effectiveness, it does not complete the job.
Level 2
Post-training tests that measure a learner's ability to retrieve information from the course are widely used in situations that require compliance (such as in the pharmaceutical industry) or technical proficiency (for certifications). Since recall is necessary to enable on the job application, post training tests are clearly a useful component when measuring the potential for impactof a sales training program, but the ability to answer questions about training materials does not necessarily translate into higher sales performance. Tests often have questions that are biased - too easy due to clues contained within the question or so difficult that learners are discouraged from getting the right answer. Online tests are often treated as "open book" exams, which in itself is not a bad thing (at least people learn how to find the correct answers!) but it does not mean that the learner can apply the knowledge, skills or information in a real sales situation.
Level 3
Measuring training transfer (Level 3) is best accomplished by an objective third party. Ideally, sales behavior in the field is observed and assessed by someone who is familiar with pre-training performance and is trained to be able to spot the differences after training. In reality, sales managers are best positioned to do this, but with a myriad of responsibilities and multiple salespeople to manage, this is rarely accomplished with a high degree of precision. Many training organizations resort to self-reported data, which predictability results in over-estimated transfer and application.
Level 4
Evaluating business impact is logistically difficult, for a number of reasons:
1. Isolating the Impact of Training
There are a variety of variables that must be controlled in order to isolate the training impact. Some are controllable - compensation and sales incentives, stabilizing territories, and keeping sales teams intact for the period of measurement. Others are uncontrollable, such as market conditions and competitive landscape.
2. Forming Control Groups
One client did all of the right things in designing a strong Level 4 evaluation, including setting up a control group of non-participants with the same tenure, experience, historical performance and baseline training. After the training, the pilot group could not help themselves but to share what they learned with the control group as they saw an immediate positive impact on their own sales results (which they attributed to the training). Thus, the control group was contaminated and the data being collected for the Level 4 evaluation was no longer valid.
3. Maintaining Vigilant Measurement
Sales organizations cannot "freeze" in the hopes of a valid scientific pursuit of impact data. The best sales teams are always tweaking, adjusting, compromising, and trying new things in order to grow their book of business. It is unreasonable to expect professionals tasked with driving revenue to ignore the very individual needs and wants of their customers to maintain precise measures of the impact of a training program. In our experience, the heavy lifting associated with measuring business results is often abandoned before the prescribed measurement period elapses.
Level 5
The holy grail of trainers is to unequivocally prove that the financial investment in training is worthwhile. The usual method is to take the fully loaded costs of training and compare those costs to the financial impact. The trick is to make sure that all costs associated with training are accounted for in the analysis. The costs that are almost always captured (due to the ease of identification) are course development costs, travel costs, logistics (shipping, printing, etc.) andthe wages and salaries of all of the staff involved in developing, delivering and attending the program (Lilly, 2001).
However, costs such as lost productivity (sales, customer response time, customer satisfaction) and administration (marketing, testing, registration, documentation, postage, conference calls) are more difficult to pinpoint and are sometimes left out of the analysis. Another commonly neglected area is the reduction in performancethat is commonly referred to as the learning curve. Performers often struggle to adopt new skills and this produces a temporary negative impact on results in the period immediately following a training event (Buelow, 2004).
Conclusion - There is Light at the End of the Tunnel
After all of these pitfalls, you may be wondering what we should do as trainers to assure our customers that it is indeed worthwhile to invest in training? The answer lies within a comprehensive strategy that encompasses all of the Levels defined here, and making incremental improvements to all levels. In upcoming posts, we will examine and recommend ways to enhance the precision for each level.
For more information on SNI's training and measurement practices, please visit www.shapironegotiations.com for a white paper - The Direct Path to Training ROI.
Comments