Impact on Organizational Performance Indicators – Level 4 Evaluation

The goal of Level 4 evaluation is to determine whether and to what extent a training course, program, or other training intervention has demonstrably improved the organization's ability to perform its mission successfully.

The concept of Level 4 evaluation has existed for over 40 years, since publication of Donald Kirkpatrick's first articles in 1959. Opportunities to conduct such evaluations successfully are generally rare; even when such opportunities do arise, they are usually accompanied by significant practical, political, and theoretical challenges.

Level 4 evaluation measures training success at the organizational level. In essence, it should be considered a major research effort and—depending on the design—may take substantial resources to accomplish. For these reasons, Level 4 evaluations are normally recommended for training programs that address a high-profile organizational issue or have a significant impact on personnel. In terms of hiring or promotion, for example, only training programs that have a direct and substantial impact on these areas should receive this level of evaluation.

Finally, Level 4 studies (based on professional conventions for research studies) are undertaken only after evaluation at Levels 1, 2, and 3 are in place and these data have been gathered, analyzed, interpreted, and acted upon. Measurement of the impact on Organizational Performance Indicators is at the heart of Level 4 Evaluation. These measurements take place after it has been determined that the organization has achieved a critical mass of individuals who have undergone Level 1, 2 and 3 evaluations, or who have "achieved competency"

PTG has developed an automated system to collect and measure, in a systematic manner, data that indicates the impact on Organizational Performance Indicators of the training event being analyzed. Data showing the impact on these indicators are given a value and compared with the cost of the training event—both direct and imputed—to estimate the worth of the training event.