Reliability analysis

P.Schaap

M.Com.

METHOD OF ANALYSIS

Selected scales from the PIB were analyzed. In the case of the PIB the sample consisted of approximately 40% Black and 60% White respondents. The VPIB comprised approximately 95% Black and 5% White respondents. The KR-20 reliability coefficient was used in the case of dichotomous scored indices and the coefficient Alpha used on the interval scaled indices. The standard errors of measurement were included where available.

DISCUSSION OF RESULTS [Refer: Tables]

The PIB and Ad-PIB seem to have respectable reliability coefficients on the indices used for the specific populations included in this study, except for Indices 22 [PIB] and 3 [Ad-VPIB] which are sub-optimal. Further analyses on other samples should be executed to verify these preliminary findings. Any final conclusion concerning the Ad-VPIB’s reliability cannot be drawn until more data is available for more comprehensive analyses. All indices seem to meet the normal distribution requirements with a slight deviation on Indices 5 and 3 on the PIB. This means that the standardized scores can be used with confidence. The reason behind these slight deviations could be sampling error, measurement error or the characteristics of the population.

The reasons for the registered distribution on Indices 3 and 5 are not clear but the nature of the group might have caused it. The standardized scores can still be used with confidence where slight deviations like these occur. Where large deviations exist, the instrument user should rely more on the raw scores until clarity is obtained concerning the reasons for the deviations. In due cause, as more candidates are assessed, clarity will be obtained on whether the deviations are due to sampling or measurement errors or due to the characteristics of the specific population. In cases where deviations are due to the characteristics of the population, standardized scores should be avoided. These may then not represent a true distribution of the raw scores. In cases where the instrument’s standard scores are based on fixed norms, calculation errors due to the inappropriate normalization of a specific group’s scores can result in decision errors.

The advantage of the PIB in this respect is the fact that raw score distributions can be monitored on a continual basis by means of the SmartStats program, with more accurate information and decisions. The actual discriminatory value estimate of an instrument based on criteria performance, can only be conclusively made if the validity of the instrument is known.