Validity in an academic setting
EMPI-I*): CONSTRUCT VALIDITY IN AN ACADEMIC INSTITUTION [1997]
INTRODUCTION
Thorndike [1986] states that the general mental ability factor, or “g”, is the main predictor of performance when mental tests are used to predict performance in school and on the job. According, to Gottfredson [1988] “g” is a very general capacity for abstract thinking, problem solving, and learning complex things. She also states that g relates highly to the speed and ease with which people acquire knowledge, especially complex knowledge, either on their own or through the instruction of others.
In the training of students, the expectation would be that these exact abilities will be the predictors of academic success. During the development of course specific selection batteries at the Technikon Pretoria instruments were included in order to assess the verbal and nonverbal reasoning abilities of candidates. These instruments are currently under scrutiny in order to determine their predictive and construct validity. In doing this, the APA’s guidelines towards the availability of situation specific validations of psychological instruments, are adhered to.
The instruments included in the mentioned batteries are:
EMPI-I
- Composition of Wholes
- Spatial Reasoning
- Perception
Academic Aptitude Test
- Verbal Reasoning
- Nonverbal Reasoning
- Spatial Perception
PIB
- Mental Alertness
- Abstract Reasoning
THE ACADEMIC APTITUDE TEST
Any number and combination of the following sub-tests from the AAT are used in the compilation of selection batteries.
- Nonverbal Reasoning (AAT 1)
- Verbal Reasoning (AAT 2)
- Spatial Perception (AAT 9)
*) later Ad-VPIB, now Scales 502, 700, 800 and 900 of SpEEx
This practice is however now being phased out and replaced by the EMPI-I. During 1996 a number of applicants were assessed with both instruments in order to compare the results statistically.
RELEVANT INDICES FROM THE EMPI-I
Composition of Wholes [Index 4]
According to the EMPI-I manual composed by Erasmus and Minnaar [1995] this index focuses on the construction of logical wholes from smaller components. It further states that the ability to order, assemble or compose is a nonverbal human attribute, as is the ability to synthesize parts into organized, integrated wholes. It is also the opinion of the developers that the capacity to compose is essentially creative in nature and involves both visual perception and logical reasoning.
Spatial Reasoning
The developers of the EMPI-I state (vide Erasmus and Minnaar, ibid.): “This process demands the nonverbal potential to perceive, analyze and synthesize. It also calls for an awareness of the relationship and ratio between objects and their parts, as well as the ability to select and to categorize.”
Perception
This index assesses the respondent’s nonverbal capacity to understand objects and situations through perception. The process also presupposes the potential to perceive detail as well as wholes (in their specific, logical and sensible context) (Erasmus and Minnaar, 1995).
THE CONSTRUCT VALIDITY OF THE EMPI-I
Correlation between the EMPI-I and the Academic Aptitude Test [AAT]
The Pearson product moment coefficients of correlation between performances on indices of the EMPI-I and scores obtained on the Academic Aptitude Test were calculated. Those performances that correlated significantly are discussed.
Index 4 [Composition of Wholes] and AAT Sub-test 1 [Nonverbal Reasoning]
The performance of the applicants for the courses in Computer Technology on Index 4 of the EMPI-I correlated significantly with their performance on Sub-test 1 of the AAT.
r = 0.4943** (**p<0.01)
The result of the correlation supports the opinion of the developers given in paragraph “Composition of Wholes [Index 4]” regarding the nonverbal abilities that are assessed by Index 4.
Index 4 [Composition of Wholes] and AAT Sub-test 9 [Spatial Perception]
The scores obtained by the candidates for the Computer Technology course on Index 4 correlated statistically significantly with their performance on Sub-test 9 of the AAT.
r = 0.4826** (**p<0.01)
This confirms Index 4 inter alia involves visual perception and logic reasoning.
Indices 6 & 7 [Spatial Reasoning and Perception] and AAT Sub-test 9 [Spatial Perception]
The performance of the applicants for the Computer Technology course on Indices 6 and 7 correlated significantly with the scores they obtained on Sub-test 9 of the AAT.
r = 0.39* (*p<0.05)
A possible explanation would be that both these tests assess the ability to perceive objects in their specific contexts.
CORRELATIONS BETWEEN EMPI-I AND PIB-INDICES
Index 4 [Composition of Wholes] and Index 5 of the PIB [Mental Alertness]
The performance of applicants for the Mechanical Engineering course on Index 4 correlates statistically significantly with their performance on Mental Alertness.
r = 0.5202** (**p<0.01)
Mental Alertness assesses abstract and general reasoning abilities which Spearman [1927] identified as the G-factor. According to Snyderman and Rothman [1986] g appears to represent an underlying and broad capacity for reasoning. The ordering of parts into a whole could be expected to demand such a reasoning ability.
Indices 6 & 7 [Spatial Reasoning and Perception] and Index 5 [Mental Alertness]
The scores obtained by candidates for the Mechanical Engineering course correlated with their performance on Mental Alertness.
r = 0.3641** (**p<0.01)
The ability to perceive, analyze and synthesize will again be underlined by the capacity for reasoning.
Index 4 [Composition of Wholes] and Index 2 [Creativity]
The performance of candidates for the Mechanical Engineering course correlated significantly with their scores on Index 2 of the PIB.
r = 0.2051* (*p<0.05)
Referring to paragraph “Composition of Wholes [Index 4]” this result confirms that the capacity to compose is essentially creative by nature.
EMPI-I INTERCORRELATIONS
The correlation between scores obtained on different EMPI-I Indices were calculated. The statistically significant results are discussed.
Index 4 [Composition of Wholes] and Indices 6 & 7 [Spatial Reasoning and Perception]
The scores obtained by candidates for the Mechanical Engineering course on Index 4 correlated significantly with the scores they obtained on Indices 6 & 7.
r = 0.4358** (**p<0.01)
In paragraphs “Index 4 [Composition of Wholes] and Index 5 of the PIB [Mental Alertness]” and “Indices 6 & 7 [Spatial Reasoning and Perception] and Index 5 [Mental Alertness]” it was said that Index 4 and Indices 6 & 7 seem to be underlined by a general capacity for reasoning. The given correlation between these indices may confirm this hypothesis.
THE CONSTRUCT VALIDITY OF THE PIB
Correlations between the PIB and the Academic Aptitude Test [AAT]
The Pearson product moment coefficients of correlation between performances on indices of the PIB and scores obtained on the Academic Aptitude Test were calculated. Those performances that correlated significantly are discussed.
Index 5 [Mental Alertness] and AAT Sub-test 1 [Nonverbal Reasoning]
The performance of the applicants for the Theatre Technology course on Index 5 correlated significantly with the scores they obtained on Sub-test 1 of the AAT.
r = 0.5817** (**p<0.01)
Mental Alertness assesses the abstract and general reasoning abilities. It can be expected that this ability will have an influence on the score obtained on a test that evaluates nonverbal reasoning abilities.
Index 5 [Mental Alertness] and AAT Sub-test 2 [Verbal Reasoning]
The scores obtained by the candidates for the Dental Assistants course on Index 5 correlated significantly with their performance on AAT 2.
r = 0.7091** (**p<0.01)
Both these tests assess the ability to reason in terms of items and materials. A highly significant correlation was therefore expected.
Index 5 and AAT Sub-test 9 [Spatial Perception]
The performance of the applicants for the Fashion Designing course on Index 5 correlated with the score they obtained on Sub-test 9 of the AAT.
r = 0.4585* (*p<0.05)
Sub-test 9 of the AAT evaluates inter alia visual perception and logic reasoning abilities. These abilities are underlined by a general reasoning ability as assessed by Index 5.
PIB-INTERCORRELATIONS
Correlations between Index 5 [Mental Alertness] and Index 7 [Abstract Reasoning] were calculated. The correlation was significant.
r = 0.4264** (**p<0.01)
It was expected that a reasoning ability will play part in the performance on both these indices. This hypothesis was confirmed.
CONCLUSION
As it is hypothesized that reasoning ability plays a significant role in academic performance, the accountable assessment of such an ability is of extreme importance. Therefore, results as these - underlining the construct validity of the assessment tools used - are invaluable in our quest to develop scientifically composed, course specific assessment batteries for selection purposes. As soon as enough data regarding the academic performance of the selected students are available, the predictive validity of these indices will be studied.
