TEST DEVELOPMENT

Putting Testing into Context

Although not a perfect science, testing is commonplace in our society. We test the products we buy, the parts we make, the materials we use, the water we drink, the air we breathe, emissions from our cars and the machines we operate. We also test people in order to drive a car, operate certain machinery, fly an airplane, get into a school, and get a job or advance within a profession. From certified public accountants to licensed electricians, the testing of people is everywhere and necessary when deciding what "proof" is needed that a person can perform to required standards and do so in a safe and efficient manner. In industry we call these tests, assessments.

WorkingSolutions offers assessment design, development and validation services to industry, apprenticeships, technical education programs and trade associations. Working with your people, processes and equipment, assessments are developed to meet the specific needs, goals, and intent of your testing program. Assessments are validated for job relatedness, effectiveness and defensibility from a legal perspective.

Before initiating an assessment process, ask the following questions:

  • What do you want to know and why?
  • Is there general agreement that the assessment will answer your questions or meet your needs?
  • If the purpose is to measure knowledge and skills, do you know what essential knowledge and skills need to be measured?
  • What will happen to those who "fail" or pass the test? What are the stakes?
  • What type of test or tests would be most appropriate?

Types of Assessments

There are many different types of assessments. Some measure very different things. Many times combinations of different assessments work best. WorkingSolutions can develop the following assessment instruments:

Written Exams (statically-based/criterion-referenced)

  • Multiple choice, best answer
  • Fill in the blanks
  • Short answer
  • Scenario based response
  • Combinations of the above

Hands-On Performance Assessments (criterion-referenced)

  • Process & product demonstrations
  • End product work samples
  • OJT checklists

Oral Assessments (objectives-based)

  • Structured panel interview

Stages of Assessment Development

  1. Job-task & skill analysis (PACE Study recommended)
  2. Table of test specifications
  3. Item bank development & crosswalk (written/oral assessments)
  4. Work activity steps & standards (performance assessments)
  5. Field test & expert review (performance assessments)
  6. Pilot test & statistical analyses (written/oral assessments)
  7. Assessment development
  8. Readability studies & bias review
  9. Cut score workshop
  10. Administration procedures

There are many ways to measure competency. Written exams can be used to test for applied knowledge and contextual skills. Interviews can be used to assess experience and critical work behaviors. Work samples demonstrate ability to produce a desired product. Performance assessments evaluate the process used and the product attained. Since some assessments involve compensation, EEO development guidelines apply as per validity, reliability, adverse impact and confidentiality. To develop assessments, a systematic approach is used to avoid arbitrary or capricious decision-making and non-empirical content.

Assessment Development Costs & Expenses

Costs: $1,200 per day plus travel expenses

All travel expenses are billed as actuals without markup. Client is responsible for selection of workgroup members, meeting room, A/V equipment and access to SOP's, existing training materials and job requirements. All assessments shall be the sole property of the client/test user. The legal and ethical use of the assessments shall be responsibility of the test user. Third-party validation of existing tests/assessments is also available. Contract days vary depending on the scope and depth of the project and the number and kind of assessments to be developed. PMA members get discounted rates.