Test Reliability – Research & Development – Pilot Study
Pilot Study (Updated)
A total of 272 new items was developed resulting in a pool of 804 items (532 items from phase 2 plus 272 new items). These items were included in a small pilot test with individuals from a variety of backgrounds (e.g. age, education, employment status). Items with extreme means, larger gender differences, or large race/ethnic differences, were removed. Priority was placed on eliminating items with similar or duplicate content. After this phase, a pool of 776 items existed.
The internal consistency estimates across all the Riasec scales were very high (ranging from .93 to .96 indicating that each of the scales “hangs together” well.
The instrument also had a high estimate of test-retest reliability (ranging from .81 to .92), providing evidence that clients’ scores are likely to be similar if the instrument is taken more than once within a short period of time
4 Primary Goals
Goal #1: Develop an instrument with strong technical characteristics that would provide clients with accurate and useful information
Goal #2: Develop a fair and unbiased instrument that would serve the needs of clients from a variety of ethnic, cultural, and socioeconomic backgrounds. Every effort was made to ensure the instrument would be helpful to clients with different backgrounds
Goal #3: Develop an instrument that included items representing the entire world of work. This would help ensure that the instrument would be useful to clients with different backgrounds
Goal #4: Develop an instrument that could be used as a self-assessment tool that individuals could self-administer, self-score, and self-interpret. This self-assessment would help empower clients to take control of their career exploration efforts. Of course, the instrument also could be used by counselors with clients in a one-on-one setting.