Psychometric Properties of the Pib SpEEx

The psychometrics properties of the SpEEx (an updated version) By Dr P Schaap Department of Human Resources Management University of Pretoria

1.1 Research objective

The objective of the research was to determine the psychometrics properties of the SpEEx for a group of employees assessed in terms of the core competencies required for specific positions in the workplace.

1.2 Data-sampling

The sample that was used for the purpose of the research is situation-specific and could therefore by referred to as a convenience sample. All available records were used at a given place and point in time. The data used for the analyses, was extracted from records within a specific company in the beverage industry.

1.3 Statistical analyses

The data was analyzed by means of Leaderware’s SmartStat’s statistical program and SPSS statistical software. Descriptive statistics were calculated and reliability analyses were performed. Effect sizes and the correlation of z-values were calculated to determine the differences in scales properties for the groups.

1.4 Results and discussion

Approximately 32% of candidates appear to have invalid results on SP1900, SP2300 and SP2400 due to missing values. The analyses were performed on valid cases only. The reliability coefficients are provided in Table 1 for the whole group. A mean coefficient has been calculated for the cognitive (0,81) and emotional (0,74) scales. Both these coefficients can be regarded as acceptable. There appears to be some variation in reliability coefficients for the different scales. Five social and emotional scales from a total of 39 scales had reliability coefficients of below 0,60. In addition, six of the social and emotional scales had reliabilities ranging between 0,60 and 0,69.

A total of 28 scales had reliabilities of 0,70 and higher. Thus, 72 % of the scales had reliabilities coefficients above the 0,70 mark that is generally considered acceptable. Eight scales from a total of 14 cognitive scales have reliabilities of 0,79 and higher. Three scales have reliabilities in the area of 0,75 and two scales have reliabilities of 0.72. A majority of the scales have reliabilities close to the more acceptable 0,80 level required for cognitive tests.

The statistics on the differences in the measurement qualities of SpEEx for Black and White population groups are provided in Table 2. In general, the cognitive tests have an average reliability of 0,81 for the Black group and 0,67 for the White group. The practical significance of the differences in reliabilities between groups is recognizable reduced when scores are normalized to a sten scale. Sten scales are one of the most commonly used scale units in the field of personality assessment. Cognisance should be taken of the fact that the average measurement error of normalized scores for both the groups has a band of error close to one sten. It is generally accepted that this value reflects appropriate levels of accuracy in the case of sten scales.

The small differences in the average measurement error (0,23) of the normalized scores between the groups, is of little practical significance. The cognitive scale measures are on the average close to being equally accurate for the groups under consideration in the case of normalized sten scores. In terms of the social and emotional scales, the Black group obtained an average reliability of 0,68 and the White group an average reliability of 0,78. As with the cognitive scales, the average measurement error of the normalized scores for the groups is close to being similar. A difference in measurement error of 0,19 (sten scale) can be regarded as of little practical significance. Given that the emotional and social scales are in English, it should be emphasized that the English linguistic abilities of the Black group would have had a considerable influence on the results of the social and emotional scale reliabilities as most candidate’s first language is not English.

The average correlation between the transformed z-values (0,89) for the cognitive scales is high, indicating the absence of construct bias for these groups. The effect sizes (difference between the mean values of raw scores) of the social and emotional scales range from small to medium. A total of six scales have medium (0,50) effect sizes and the remaining 33 scales have small effect sizes. Furthermore, the average effect size for the emotional and social scales is 0,22. Thus, the difference in the mean values of the raw scores for these scales is one fifth of a standard deviation and is unlikely to have a practical significant impact on the selection of employees in general.

In the last instance, it should be emphasized that the Leadership Styles inventory’s Transformational and Transactional Leadership total scores should be the basis on which interpretations are made and not the sub-scales. As indicated in Table 1, the reliability coefficients are acceptable for both scales. The sub-scales were not included in Table 1 for this reason. The results in Table 3 and Table 4 clearly indicate that the sub-scales should not be used as independent measures. The reliabilities of the sub-scales are generally unacceptably low, and should only be used for a more detailed explanation of the Leadership Style Scales.

It is important to realize that the discussion on the comparison between the scale properties for the population groups is based on the average value for the different scales. The author is by no means trying to convey that scale reliabilities that are high for specific scales in SpEEx, compensate for scales that have less acceptable scale reliabilities. A more appropriate approach would be to evaluate each scale at its own merit, depending on the type and importance of the scale and the context in which it is used. The purpose of this discussion is to provide a general overview of the average measurement properties of the SpEEx as a whole. Secondly, it is important to realize that the conclusions that are made concerning the practical significance of the differences in scale measurement accuracies, is based on the scale properties of normalized sten scores and do not necessarily apply to raw scores or other normalized scores with more scale intervals.

SpEEX Stats	N	Items	Mean	SD	Reliability
Conceptualization	1689	30	20.17	6.24	0.9
Memory	1177	20	12.41	3.65	0.75
Basic Calculations	1099	20	12.2	5.46	0.91
Advanced Calculations	1062	20	8.9	3.41	0.72
Observance	1427	25	14.14	3.45	0.76
Assembling (Basic)	863	-	6.55	1.55	-
Assembling (Advanced)	631	14	6.72	3.86	0.85
Clerical	900	20	13.65	4.98	0.9
Comparison	1164	10	12.72	4.39	0.79
Perception	677	10	6.7	2.54	0.75
Reading Comprehension	264	20	6.69	4.5	0.85
Listening Potential	245	20	7.04	3.28	0.72
Linguistic Proficiency (Basic)	1015	10	13.17	3.83	0.79
Linguistic Proficiency (Adv)	522	16	13.22	5.39	0.89
Cognitive Average					0.81
Insight	338	13	8.32	3.13	0.66
Socialisation	365	20	91.64	17.62	0.89
Adaptability	677	20	64.2	14.59	0.69
Internal Actualization	255	20	82.73	18.59	0.89
External Actualization	127	20	62.06	23.09	0.88
Avertion	445	20	60.53	14.47	0.7
Contest	445	20	57.92	20.25	0.85
Negotiation	445	20	99.06	18.1	0.9
Compliance	445	20	68.38	17.33	0.82
Compromise	445	20	75.36	18.56	0.84
Self-actualisation	230	20	84.52	18.46	0.88
Demonstrative	488	10	8.78	3.52	0.56
Samaritarian	488	10	12.72	4.24	0.58
Persevering	488	10	8.6	3.31	0.49
Evaluating	488	10	9.81	3.46	0.48
Conformity/Non-conform	234	20	72.71	15.32	0.72
Empathy	273	12	49.24	11.55	0.8
Emotional sensitivity	273	12	50.38	11.21	0.83
Tact	273	12	54.34	10.82	0.83
People Development	273	12	61.81	10.64	0.86
Mental Stress	273	12	53.89	12.47	0.8
Interpersonal Object	273	12	53.92	10.15	0.76
Physical Stress	273	12	62.42	11.28	0.82
Diversity Facilitation	273	12	34.74	9.52	0.63
Transactional leadership	1025	48	52.39	17.35	0.79
Transformational leadership	1025	48	57.32	17	0.76
Excellence Orientation	1212	8	9.65	4.17	0.8
Customer Orientation	1212	8	11.33	3.56	0.74
Innovation	1212	8	11.51	2.91	0.57
Feedback	1212	8	8.65	3.78	0.73
Presentation	1212	8	5.08	3.52	0.75
Negotiation	1212	8	6.67	3.29	0.66
Liaison	1212	8	5.82	3.66	0.77
Analytical Thinking	1212	8	7.01	3.19	0.62
Judgement	1212	8	6.22	3.11	0.63
Organizational Alertness	1212	8	6.06	3.4	0.7
Nonverbal Perception	1212	8	6.42	3.49	0.72
Personal Development	1212	9	12.68	3.59	0.71
Written Communication	1212	8	7.83	3.67	0.73
Emotional & Social Average					0.74

Table 2: Statistics on differences in the measurement qualities of SpEEx for Black and White groups

SpEEx Stats		Black group			White group			Differences
SCALE	Items	N.b	Rtt.b	Sten Sem.b	N.w	Rtt.w	Sten Sem.w	StenSem.d	Corr/Eff
Conceptualization (SP100)	30.00	970	0.89	0.66	364	0.80	0.88	0.22	0.94*
Memory	20.00	735	0.74	1.00	236	0.64	1.20	0.20	0.89*
Basic Calculations	20.00	0.91	-	-	-
Advanced Calculations	20.00	572	0.65	1.18	219	0.55	1.34	0.16	0.94*
Observance	25.00	1,062	0.75	1.00	201	0.56	1.32	0.32	0.93*
Assembling (Advanced)	14.00	332	0.83	0.82	150	0.85	0.76	0.06	0.86*
Clerical	20.00	548	0.89	0.66	185	0.65	1.18	0.52	0.90*
Comparison	10.00	650	0.73	1.04	300	0.77	0.94	0.10	0.80*
Linguistic Proficiency (Adv)	16.00	652	0.88	-	215	0.53	-
Cognitive Averages			0.81	0.91		0.67	1.09	0.23	0.89
Adaptability	20.00	315	0.59	1.28	111	0.77	0.94	0.34	0.12
Aversion	100.00	252	0.68	1.12	111	0.73	1.04	0.08	0.04
Contest	100.00	252	0.82	0.84	111	0.87	0.72	0.12	0.09
Negotiation	100.00	252	0.89	0.66	111	0.90	0.64	0.02	0.25
Compliance	100.00	252	0.80	0.88	111	0.85	0.78	0.10	0.01
Compromise	100.00	252	0.83	0.82	111	0.86	0.74	0.08	0.01
Demonstrative	40.00	300	0.51	1.40	110	0.69	1.11	0.20	0.01
Samaritarian	40.00	300	0.54	1.35	110	0.74	1.00	0.26	0.33
Persevering	40.00	300	0.41	1.53	111	0.54	1.46	0.42	-0.3
Evaluating	40.00	300	0.39	1.56	111	0.58	1.30	0.32	-0.15
Empathy	12.00	119	0.77	0.94	99	0.86	0.74	0.20	-0.1
Emotional sensitivity	12.00	119	0.82	0.84	99	0.86	0.74	0.10	-0.11
Tact	12.00	119	0.83	0.82	99	0.84	0.80	0.02	0.03
People Development	12.00	119	0.86	0.74	99	0.88	0.68	0.06	0.09
Mental Stress	12.00	119	0.78	0.92	99	0.81	0.86	0.06	-0.57
Interpersonal Object	12.00	119	0.74	1.02	99	0.81	0.86	0.16	0.01
Physical Stress	12.00	119	0.82	0.84	99	0.82	0.84	0.00	-0.54
Diversity Facilitation	12.00	119	0.64	1.20	99	0.55	1.34	0.14	0.66
Transactional leadership	48.00	492	0.75	1.00	305	0.81	0.86	0.14	0.47
Transformational leadership	48.00	492	0.74	1.02	305	0.80	0.89	0.13	0.29
Excellence Orientation	8.00	652	0.73	1.04	290	0.84	0.80	0.23	-0.44
Customer Orientation	8.00	652	0.68	1.13	290	0.80	0.89	0.24	0.11
Innovation	8.00	652	0.53	1.37	290	0.66	1.16	0.19	-0.26
Feedback	8.00	652	0.67	1.15	290	0.82	0.84	0.30	0.17
Presentation	9.00	652	0.65	1.18	290	0.83	0.82	0.36	0.61
Negotiation	8.00	652	0.57	1.31	290	0.77	0.95	0.35	-0.17
Liaison	7.00	652	0.69	1.11	290	0.81	0.87	0.24	0.18
Analytical Thinking	8.00	652	0.49	1.43	290	0.73	1.03	0.38	-0.01
Judgement	8.00	652	0.52	1.39	290	0.73	1.03	0.34	0.16
Organizational Alertness	8.00	652	0.65	1.18	290	0.72	1.05	0.12	0.56
Nonverbal Perception	8.00	652	0.62	1.23	290	0.78	0.93	0.29	-0.01
Personal Development	9.00	652	0.68	1.13	290	0.70	1.09	0.03	-0.4
Written Communication	8.00	652	0.66	1.16	290	0.82	0.84	0.32	-0.36
Social & Emotional Averages			0.68	1.11		0.78	0.93	0.19	0.22

Abbreviations:

Sem.b = Standard error of measurement black group (Sten scale)
Nb = sample black group
Sem.w = Standard error of measurement white group (Sten scale)
Nw = sample white group
Sem.d = Difference in standard error of measurement for groups
Rtt.b = Scale reliability black group
Corr/Eff = Correlation between z-scores and effect sizes
Rtt.w = Scale reliability white group
= Marks for correlation between z-scores

Table 3: Reliability statistics for Leadership Style subscales (SP2400)