The psychometrics properties of the SpEEx (an updated version) By Dr P Schaap Department of Human Resources Management University of Pretoria
1.1 Research objective
The objective of the research was to determine the psychometrics properties of the SpEEx for a group of employees assessed in terms of the core competencies required for specific positions in the workplace.
The sample that was used for the purpose of the research is situation-specific and could therefore by referred to as a convenience sample. All available records were used at a given place and point in time. The data used for the analyses, was extracted from records within a specific company in the beverage industry.
1.3 Statistical analyses
The data was analyzed by means of Leaderware�s SmartStat�s statistical program and SPSS statistical software. Descriptive statistics were calculated and reliability analyses were performed. Effect sizes and the correlation of z-values were calculated to determine the differences in scales properties for the groups.
1.4 Results and discussion
Approximately 32% of candidates appear to have invalid results on SP1900, SP2300 and SP2400 due to missing values. The analyses were performed on valid cases only. The reliability coefficients are provided in Table 1 for the whole group. A mean coefficient has been calculated for the cognitive (0,81) and emotional (0,74) scales. Both these coefficients can be regarded as acceptable. There appears to be some variation in reliability coefficients for the different scales. Five social and emotional scales from a total of 39 scales had reliability coefficients of below 0,60. In addition, six of the social and emotional scales had reliabilities ranging between 0,60 and 0,69.
A total of 28 scales had reliabilities of 0,70 and higher. Thus, 72 % of the scales had reliabilities coefficients above the 0,70 mark that is generally considered acceptable. Eight scales from a total of 14 cognitive scales have reliabilities of 0,79 and higher. Three scales have reliabilities in the area of 0,75 and two scales have reliabilities of 0.72. A majority of the scales have reliabilities close to the more acceptable 0,80 level required for cognitive tests.
The statistics on the differences in the measurement qualities of SpEEx for Black and White population groups are provided in Table 2. In general, the cognitive tests have an average reliability of 0,81 for the Black group and 0,67 for the White group. The practical significance of the differences in reliabilities between groups is recognizable reduced when scores are normalized to a sten scale. Sten scales are one of the most commonly used scale units in the field of personality assessment. Cognisance should be taken of the fact that the average measurement error of normalized scores for both the groups has a band of error close to one sten. It is generally accepted that this value reflects appropriate levels of accuracy in the case of sten scales.
The small differences in the average measurement error (0,23) of the normalized scores between the groups, is of little practical significance. The cognitive scale measures are on the average close to being equally accurate for the groups under consideration in the case of normalized sten scores. In terms of the social and emotional scales, the Black group obtained an average reliability of 0,68 and the White group an average reliability of 0,78. As with the cognitive scales, the average measurement error of the normalized scores for the groups is close to being similar. A difference in measurement error of 0,19 (sten scale) can be regarded as of little practical significance. Given that the emotional and social scales are in English, it should be emphasized that the English linguistic abilities of the Black group would have had a considerable influence on the results of the social and emotional scale reliabilities as most candidate�s first language is not English.
The average correlation between the transformed z-values (0,89) for the cognitive scales is high, indicating the absence of construct bias for these groups. The effect sizes (difference between the mean values of raw scores) of the social and emotional scales range from small to medium. A total of six scales have medium (0,50) effect sizes and the remaining 33 scales have small effect sizes. Further more, the average effect size for the emotional and social scales is 0,22. Thus, the difference in the mean values of the raw scores for these scales is one fifth of a standard deviation and is unlikely to have a practical significant impact on the selection of employees in general.
In the last instance, it should be emphasized that the Leadership Styles inventory�s Transformational and Transactional Leadership total scores should be the basis on which interpretations are made and not the subscales. As indicated in Table 1, the reliability coefficients are acceptable for both scales. The subscales were not included in Table 1 for this reason. The results in Table 3 and Table 4 clearly indicate that the subscales should not be used as independent measures. The reliabilities of the subscales are generally unacceptably low, and should only be used for a more detailed explanation of the Leadership Style Scales.
It is important to realize that the discussion on the comparison between the scale properties for the population groups is based on the average value for the different scales. The author is by no means trying to convey that scale reliabilities that are high for specific scales in SpEEx, compensate for scales that have less acceptable scale reliabilities. A more appropriate approach would be to evaluate each scale at its own merit, depending on the type and importance of the scale and the context in which it is used. The purpose of this discussion is to provide a general overview of the average measurement properties of the SpEEx as a whole. Secondly, it is important to realize that the conclusions that are made concerning the practical significance of the differences in scale measurement accuracies, is based on the scale properties of normalized sten scores and do not necessarily apply to raw scores or other normalized scores with more scale intervals.
|Linquistic Proficiency (Adv)||522||16||13.22||5.39||0.89|
|Emosional & Social Average||0.74|
Table 2: Statistics on differences in the measurement qualities of SpEEx for Black and White groups
|SpEEx STATS||Black group||White group||Differences|
|SCALE||Items||N.b||Rtt.b||Sten Sem.b||N.w||Rtt.w||Sten Sem.w||StenSem.d|
|Linquistic Proficiency (Adv)||16.00||652||0.88||-||215||0.53||-|
|Social & Emotional Averages||0.68||1.11||0.78||0.93||0.19||0.22|
Table 3: Reliability statistics for Leadership Style subscales (SP2400)