Psychometric properties of the PibSpeex


Psychometric properties of the PibSpeex

    The psychometrics properties of the SpEEx (an updated version) By Dr P Schaap Department of Human Resources Management University of Pretoria

1.1 Research objective

The objective of the research was to determine the psychometrics properties of the SpEEx for a group of employees assessed in terms of the core competencies required for specific positions in the workplace.

1.2 Data-sampling

The sample that was used for the purpose of the research is situation-specific and could therefore by referred to as a convenience sample. All available records were used at a given place and point in time. The data used for the analyses, was extracted from records within a specific company in the beverage industry.

1.3 Statistical analyses

The data was analyzed by means of Leaderware�s SmartStat�s statistical program and SPSS statistical software. Descriptive statistics were calculated and reliability analyses were performed. Effect sizes and the correlation of z-values were calculated to determine the differences in scales properties for the groups.

1.4 Results and discussion

Approximately 32% of candidates appear to have invalid results on SP1900, SP2300 and SP2400 due to missing values. The analyses were performed on valid cases only. The reliability coefficients are provided in Table 1 for the whole group. A mean coefficient has been calculated for the cognitive (0,81) and emotional (0,74) scales. Both these coefficients can be regarded as acceptable. There appears to be some variation in reliability coefficients for the different scales. Five social and emotional scales from a total of 39 scales had reliability coefficients of below 0,60. In addition, six of the social and emotional scales had reliabilities ranging between 0,60 and 0,69.

A total of 28 scales had reliabilities of 0,70 and higher. Thus, 72 % of the scales had reliabilities coefficients above the 0,70 mark that is generally considered acceptable. Eight scales from a total of 14 cognitive scales have reliabilities of 0,79 and higher. Three scales have reliabilities in the area of 0,75 and two scales have reliabilities of 0.72. A majority of the scales have reliabilities close to the more acceptable 0,80 level required for cognitive tests.

The statistics on the differences in the measurement qualities of SpEEx for Black and White population groups are provided in Table 2. In general, the cognitive tests have an average reliability of 0,81 for the Black group and 0,67 for the White group. The practical significance of the differences in reliabilities between groups is recognizable reduced when scores are normalized to a sten scale. Sten scales are one of the most commonly used scale units in the field of personality assessment. Cognisance should be taken of the fact that the average measurement error of normalized scores for both the groups has a band of error close to one sten. It is generally accepted that this value reflects appropriate levels of accuracy in the case of sten scales.

The small differences in the average measurement error (0,23) of the normalized scores between the groups, is of little practical significance.  The cognitive scale measures are on the average close to being equally accurate for the groups under consideration in the case of normalized sten scores. In terms of the social and emotional scales, the Black group obtained an average reliability of 0,68 and the White group an average reliability of 0,78. As with the cognitive scales, the average measurement error of the normalized scores for the groups is close to being similar. A difference in measurement error of 0,19 (sten scale) can be regarded as of little practical significance. Given that the emotional and social scales are in English, it should be emphasized that the English linguistic abilities of the Black group would have had a considerable influence on the results of the social and emotional scale reliabilities as most candidate�s first language is not English.

The average correlation between the transformed z-values (0,89) for the cognitive scales is high, indicating the absence of construct bias for these groups. The effect sizes (difference between the mean values of raw scores) of the social and emotional scales range from small to medium. A total of six scales have medium (0,50) effect sizes and the remaining 33 scales have small effect sizes. Further more, the average effect size for the emotional and social scales is 0,22. Thus, the difference in the mean values of the raw scores for these scales is one fifth of a standard deviation and is unlikely to have a practical significant impact on the selection of employees in general.

In the last instance, it should be emphasized that the Leadership Styles inventory�s Transformational and Transactional Leadership total scores should be the basis on which interpretations are made and not the subscales. As indicated in Table 1, the reliability coefficients are acceptable for both scales. The subscales were not included in Table 1 for this reason. The results in Table 3 and Table 4 clearly indicate that the subscales should not be used as independent measures. The reliabilities of the subscales are generally unacceptably low, and should only be used for a more detailed explanation of the Leadership Style Scales.

It is important to realize that the discussion on the comparison between the scale properties for the population groups is based on the average value for the different scales. The author is by no means trying to convey that scale reliabilities that are high for specific scales in SpEEx, compensate for scales that have less acceptable scale reliabilities. A more appropriate approach would be to evaluate each scale at its own merit, depending on the type and importance of the scale and the context in which it is used. The purpose of this discussion is to provide a general overview of the average measurement properties of the SpEEx as a whole. Secondly, it is important to realize that the conclusions that are made concerning the practical significance of the differences in scale measurement accuracies, is based on the scale properties of normalized sten scores and do not necessarily apply to raw scores or other normalized scores with more scale intervals.

SpEEX STATSNItemsMeanSDReliability

Conceptualization

16893020.176.24 0.9
Memory11772012.413.650.75
Basic Calculations10992012.25.460.91
Advanced Calculations1062208.93.410.72
Observance14272514.143.450.76
Assembling (Basic)863-6.551.55-
Assembling (Advanced)631146.723.860.85
Clerical9002013.654.980.9
Comparison11641012.724.390.79
Perception677106.72.540.75
Reading Comprehension264206.694.50.85
Listening Potential245207.043.280.72
Linquistic Proficiency(Basic)10151013.173.83 0.79
Linquistic Proficiency (Adv)5221613.22 5.390.89
Cognitive Average    0.81
Insight338138.323.13 0.66
Socialisation3652091.6417.620.89
Adaptability6772064.214.590.69
Internal Actualization2552082.73 18.590.89
External Actualization1272062.0623.090.88
Avertion4452060.5314.47 0.7
Contest4452057.9220.250.85
Negotiation4452099.06 18.1 0.9
Compliance4452068.3817.33 0.82
Compromise4452075.3618.56 0.84
Self-actualisation2302084.5218.460.88
Demonstrative488108.783.520.56
Samaritarian4881012.724.240.58
Persevering488108.63.310.49
Evaluating488109.813.460.48
Conformity/Non-conform2342072.7115.320.72
Empathy2731249.2411.550.8
Emotional sensitivity27312 50.38 11.210.83
Tact2731254.3410.820.83
People Development2731261.8110.640.86
Mental Stress2731253.8912.470.8
Interpersonal Object2731253.9210.15 0.76
Physical Stress2731262.4211.280.82
Diversity Facilitation2731234.749.520.63
Transactional leadership10254852.3917.35 0.79
Transformational leadership10254857.32170.76
Excellence Orientation12128 9.654.17 0.8
Customer Orientation1212811.333.560.74
Innovation1212811.512.910.57
Feedback12128 8.653.78 0.73
Presentation121285.08 3.520.75
Negotiation121286.673.290.66
Liaison121285.823.660.77
Analytical Thinking121287.013.190.62
Judgement121286.223.110.63
Organizational Alertness121286.06 3.40.7
Nonverbal Perception121286.423.490.72
Personal Development1212912.683.59 0.71
Written Cummunication121287.83 3.670.73
Emosional & Social Average    0.74

Table 2: Statistics on differences in the measurement qualities of SpEEx for Black and White groups

SpEEx STATS Black groupWhite groupDifferences
SCALEItemsN.bRtt.b Sten Sem.bN.wRtt.wSten Sem.wStenSem.d

Corr/Eff

Conceptualization (SP100)30.009700.890.663640.800.880.220.94*
Memory20.007350.741.002360.641.200.20  0.89*
Basic Calculations20.000.91---



Advanced Calculations20.005720.651.182190.551.340.160.94*
Observance25.001,0620.751.002010.561.320.320.93*
Assembling (Advanced)14.003320.830.821500.85 0.760.060.86*
Clerical20.005480.890.66 1850.651.180.520.90*
Comparison10.006500.731.043000.770.940.100.80*
Linquistic Proficiency (Adv)16.006520.88 -2150.53-

Cognitive Averages    

0.810.91
0.671.090.230.89
Adaptability20.003150.591.281110.770.940.340.12
Avertion100.002520.681.121110.731.040.080.04
Contest100.002520.82 0.841110.87 0.720.120.09
Negotiation100.002520.890.66111 0.900.640.02 0.25
Compliance100.002520.80 0.881110.850.780.10 0.01
Compromise100.002520.830.821110.860.740.08 0.01
Demonstrative40.003000.511.401100.691.110.200.01
Samaritarian40.003000.541.351100.741.000.260.33
Persevering40.003000.41 1.531110.541.460.42-0.3
Evaluating40.003000.391.561110.581.300.32-0.15
Empathy12.001190.77 0.94990.860.740.20-0.1
Emotional sensitivity 12.001190.820.84990.860.740.10 -0.11
Tact12.00119 0.830.82990.840.800.020.03
People Development12.001190.860.74990.880.680.060.09
Mental Stress12.001190.78 0.92990.810.860.06-0.57
Interpersonal Object12.00119 0.741.0299 0.810.860.160.01
Physical Stress12.001190.820.8499 0.820.840.00-0.54
Diversity Facilitation12.001190.64 1.20990.551.340.14 0.66
Transactional leadership48.00492 0.751.003050.810.860.14 0.47
Transformational leadership48.004920.74 1.02305 0.800.890.130.29
Excellence Orientation8.00 652 0.73 1.04 290 0.840.80 0.23-0.44
Customer Orientation8.00 652 0.681.132900.80 0.89 0.24 0.11
Innovation8.00 652 0.53 1.37290 0.66 1.160.19-0.26
Feedback 8.00 6520.671.152900.820.840.300.17
Presentation9.00 6520.651.182900.830.82   0.36 0.61
Negotiation8.00 6520.571.312900.77 0.950.35-0.17
Liaison7.006520.691.112900.810.870.24 0.18
Analytical Thinking8.00 6520.49 1.432900.73 1.030.38-0.01
Judgement8.00 6520.52 1.392900.73 1.030.340.16
Organizational Alertness8.00 6520.65 1.18290 0.72 1.05 0.12 0.56
Nonverbal Perception 8.00 652 0.62 1.232900.780.930.29-0.01
Personal Development 9.00652 0.681.132900.701.090.03-0.4
Written Cummunication8.006520.661.162900.820.84 0.32-0.36
Social & Emotional Averages

0.681.11
 0.780.930.19 0.22

  

 

Abbreviations:

  • Sem.b = Standard error of measurement black group (Sten scale)
  • Nb = sample black group
  • Sem.w = Standard error of measurement white group (Sten scale)
  • Nw = sample white group
  • Sem.d = Difference in standard error of measurement for groups
  • Rtt.b = Scale reliability black group
  • Corr/Eff = Correlation between z-scores and effect sizes
  • Rtt.w = Scale reliability white group
  • * = Marks for correlation between z-scores

Table 3: Reliability statistics for Leadership Style subscales (SP2400)

 

 

This site was built using