Level 2 Online Test Construction

Level 2 (Achievement) Evaluation
A well-designed training program must enable students to return to the job following their training and successfully apply the knowledge and skills taught or learned during the training program. Achieving this goal requires completion of a thorough job task analysis and development of course objectives. Test items are then written that measure whether or not the students have accomplished the course objectives.

If there is no measure of accomplishment, there is no way to document the worth of the training event. However, for training worth to be measured properly the test must be valid, and it must also produce reliable test results. The lack of these characteristics diminishes the value of testing, per se, and can even have legal ramifications. The method selected for reliability determination is determined based on the structure (tests with a range of scores vs. binary tests to determine mastery/non-mastery), and use (recruitment, promotion, separation, course improvement, trainee progress development) of the test.

Tests developed using PTG's automated approach to job task analysis, objective determination, and analysis of test items, are assumed to have met the requirements for face and content validity. The system also makes available appropriate coefficients to calculate and determine criterion-related validity, if appropriate, for the planned use of the test.

PTG has developed an online system for test development and reliability verification, called TDRS. TDRS offers an automated methodology for developing/isolating job tasks. TDRS provides a method for administering pilot tests online, or for importing the results of paper-based pilot tests. Once job tasks are established, an automated "wizard" walks the test developer through a series of worksheets for performing job task analysis and objective determination, and for review and analysis of the test items themselves. Test items, that meet the criteria in the "wizard", are assumed to have met the requirements for Face and Content validity. Worksheets are available to calculate coefficients needed to establish criterion-related validity. TDRS includes a facility for review of each worksheet's results by an outside expert.

Tests developed and validated as described above, are stored in a test repository, for later administration. Once the test has been administered and scores are available, a Test Reliability Estimation System is used to estimate the reliability of the test results overall, based on the actual test results.