Predictive and construct validity

HELENA KRIEL 

M.A. 

 

INTRODUCTION

Selecting students for tertiary institutions is a worldwide problem. According to Stoker, Engelbrecht, Crowther, Du Toit and Herbst [1985] a study by Malherbe and Cook in 1938 was the first in this area in South Africa. Almost sixty years later the issue of admission policy and predicting academic performance on tertiary level has become even more complicated.

Traditionally, scholastic achievements were used as selection criteria to gain access to South African tertiary institutions. Kotze, Van der Merwe and Nel [1996] state that mainly because of the unequal educational opportunities, [this] placed many students at a severe disadvantage. Although research in South Africa and elsewhere has clearly indicated that matriculation results remain the best predictor of academic success at tertiary level, it was also found that matriculation results of matriculants from the previous Department of Education and Training and equivalent school systems, particular at the lower ranges, are an inaccurate reflection of students’ academic ability, or potential for success at tertiary level.

An alternative selection procedure was therefore inevitable, as the Technikon Pretoria, as all other tertiary institutions cannot rely on unreliable indicators of academic performance. The focus had to move to assessing the individual’s potential - both manifested and latent. Stoker, Engelbrecht, Crowther, Du Toit and Herbst [1985] recommended course specific selection. The fields of studies at universities, but especially at technikons, impose distinctive requirements of and challenges to the students. The ideal therefore would be a specific selection procedure or differential admission requirements for each course of study.

With this in mind the Bureau for Student Counseling, who is responsible for the assessing of the potential of prospective students at the Technikon Pretoria, started to use the Potential Index Battery in April 1996. Since then more than 5000 applicants, for 45 different courses, have been evaluated. Research is now being done on these specific batteries of tests, in order to validate the factors hypothesized as being predictors of academic success. This is done by evaluating senior students with the specific batteries for their courses, as well as by monitoring the students selected by means of the potential assessment batteries.


THE RANGE OF PSYCHOMETRIC INSTRUMENTS USED BY THE TECHNIKON

PRETORIA IN THE ASSESSMENT OF POTENTIAL

Different combination of PIB-indices, the Brown-Holtzman Survey of Study Habits and Attitudes [SSHA] and the Academic Aptitude Test [AAT] are used in assessing potential students. The Brown-Holtzman Survey of Study Habits and Attitudes [SSHA] was developed to evaluate study habits and attitudes that relate with academic performance. It consists of a hundred questions that measure the candidate’s study orientation on four sub-scales, namely:

  • Avoidance of Delay
  • Work Methods
  • Education Acceptance
  • Teacher Acceptance

As shown later in this report, scores obtained on this questionnaire correlates significantly with certain PIB-indices. In some cases, any number of the following three sub-tests of the Academic Aptitude Test [AAT] are included in the potential assessment batteries:

  • Non-Verbal Reasoning
  • Verbal Reasoning
  • Spatial Reasoning

This practice is now being phased out and the Visual PIB [EMPI] is considered as the probable alternative.


PIB RESULTS

PIB-Indices as predictors of academic success

The Pearson product moment coefficients of correlation between scores obtained on Indices of the PIB and academic performances were calculated. Mathematically calculated means of test and exam results were used as indicators of academic performance. Those indices that correlated significantly with academic performance are discussed.


Index 5 [Mental Alertness]

The performance of the students from the Finesse course on this index correlated statistically significantly with their academic performance.

r = 0.574* (*p<0.05)

From previous research [Schaap, 1996] it seems that Index 5 measures abstract and general reasoning abilities. These abilities seem to be predictors of academical success for this specific course.


Correlation between PIB and other psychometric instruments

The Pearson product moment coefficients of correlation between performances on Indices of the PIB and scores obtained on other psychometric instruments were calculated. Those performances that correlated significantly are discussed.


Index 9 [Interpersonal Relations] and the Survey of Study Habits and Attitudes

[SSHA]

The performance of first year Library and Information Technology students on Index 9 correlated statistically significantly with their performance on the SSHA.

r = 0.5842* (*p<0.05)

A person with well-developed interpersonal relations is a person who accepts authority readily. The SSHA evaluated the student’s acceptance of teachers/lecturers which may explain the correlation between the performance of students on Index 9 and SSHA.


Index 15 [Self-motivation] and the Survey of Study Habits and Attitudes [SSHA]

The performance of first year Journalism students on Index 15 and on the SSHA correlated statistically significantly.

r = 0.6228** (**pp<0.01)

The Brown-Holtzman Survey of Study Habits and Attitudes [SSHA] evaluated the student’s attitudes towards his studies. It can be expected that an internally motivated person with a sense of responsibility and the ability to function autonomously will have a positive attitude towards his/her studies.


Index 18 [Stress Management] and the Survey of Study Habits and Attitudes [SSHA]

The scores obtained by first, second and final year Drama students on Index 18 of the PIB correlated statistically significantly with their performance on the SSHA.

r = 0.5423* (*p<0.05)

A possible explanation would be that a well-organized, methodical student will not as easily feel overloaded as a student whose work methods are hap hazardous.


Index 24 [Coping Skills] and the Survey of Study Habits and Attitudes [SSHA]

The performance of first year Journalism students on Index 24 correlated statistically highly significantly with the scores they obtained on the SSHA.

r = 0.6922** (**p<0.01)

A student who is positive and optimistic and handles pressure well, will more likely than not be a student with a high level of acceptance of lectures and education. It will also be a student who does not tend to procrastinate.


PIB-Intercorrelations

The correlation between scores obtained on different PIB-indices were calculated. Those indices that correlated with a statistical significance are discussed.


Index 2 [Creativity] and Index 17 [Time Management]

The performance of Journalism students on Index 2 and Index 17 correlated statistically significantly.

r = 0.7281** (**p<0.01)

A creative person is an unconventional, unsystematic person who is not structure bound and enjoys crisis management. This will not be a person who is high on time management. This seems to be an ideal combination in Journalism students.


Index 15 [Motivation] and Index 17 [Time Management]

The scores obtained by Journalism students on Indices 15 and 17 correlated statistically significantly.

r = 0.5217* (*p<0.05)

A high score on Index 15 indicates a person with an internal locus of control, self-discipline and a sense of responsibility. Such a person will manage his/her time well.


Index 17 [Time Management] and Index 18 [Stress Management]

The performance of first year Journalism students on Indices 17 and 18 correlated statistically significantly.

r = 0.6207** (**p<0.01)

A possible explanation could be that a person who manages time well will not experience overload and an important source of stress is thus eliminated.

 

Index 9 [Interpersonal Relations] and Index 10 [Self-Image]

The scores obtained by Journalism students on above mentioned indices correlated statistically significantly.

r = 0.7085** (**p<0.01)

These results support the traditional hypothesis that self-acceptance leads to the acceptance of others. The well-known phrase of “I’m OK; You’re OK” is applicable in this situation.


Index 10 [Self-Image] and Index 15 [Motivation]

The performance of Journalism students on Index 10 and Index 15 correlated with statistical significance.

r = 0.6915** (**p<0.01)

A person with an internal locus of control will be less dependent on the approval of others and will have more confidence in his/her own abilities and judgement. Such a person will present a well-developed image.


Index 10 [Self-Image] and Index 17 [Time Management]

The scores obtained by Journalism students on Index 10 correlated with their performance on Index 17.

r = 0.6281** (**p<0.01)

This might be explained by saying that a person who is confident in his/her own abilities and judgement and who is not dependent on the approval of others will also be able to take control of his/her own time.


Index 10 [Self-Image] and Index 18 [Stress Management]

The performance of first, second and final year Drama students on Index 10 correlated statistically significantly with their performance on Index 18.

r = 0.7071** (**p<0.01)

A negative self-image is most likely the largest single internal source of stress. A person who presents a negative self-image tends towards constant self-criticism and discrediting. Such a person feels overwhelmed by environmental factors and as if he/she has lost control over his/her life. Psychosomatic symptoms are frequent in people with negative self-esteems.


Index 10 [Self-Image] and Index 24 [Coping Skills]

The performance of Journalism students on Index 10 correlated statistically significantly with their performance on Index 24.

r = 0.6530** (**p<0.01)

People with negative self-images tend to be more pessimistic in their attitude towards life and have more difficulty in handling pressure than people with positive self-images.


Index 18 [Stress management] and Index 24 [Coping Skills]

The scores obtained by third year Radiography students on Index 18 correlated with their scores on Index 24.

r = 0.6952* (*p<0.05)

A person who tends to be optimistic and handles pressure well, will most probably be a person who manages stress well.


PIB-PROFILES FOR SPECIFIC COURSES

The main aim of this research project done by the Bureau for Student Counseling, is the compilation of a profile for each specific course presented at the Technikon Pretoria. In order to do this, preliminary profiles have been compiled. It is extremely important to realize that these are only preliminary profiles as the sample groups were small and such a profile cannot be compiled by using once off test results. These profiles, however, provide us with the starting blocks that we so dearly need in order to get the desired course specific selection procedures in place. It is interesting though to have a look at an example of such a course specific profile.

For the sake of this article, the profile for the Drama course was chosen. Frequency tables were compiled and are given graphically in order to show how the distributions of scores differ from what one would expect from an “every day, every course” student. Scores obtained by Radiography students are given for the comparison purposes.


CONCLUSION

Although the showed graphs cannot be interpreted as the final profile for the mentioned courses, the argument introduced by Stoker et al [1985] for specific selection criteria for specific courses, seems to be supported. It also seems that not only the application of a certain combination of indices, but also the interpretation of the scores obtained on indices, will differ from course to course. It is also clear that it will take thoroughly executed, scientifically funded research to develop specific selection criteria for each course presented at the Technikon Pretoria. The PIB seems to be an extremely useful recourse.


LIST OF REFERENCES

Kotze, N,; Van der Merwe, D & Nel A. (1996) Culture fair selection procedures: The case of psychometrics. Paper presented at the SSCA conference PREMOS. Schaap, P. (1996) The predictive and construct validity of the PIB in a financial institution. Unpublished report.

Stoker, D.J.; Engelbrecht, C.S.; Crowther, N.A.S.; Du Toit, S.H.C. & Herbst, A. (1985). Ondersoek na differensiële toelatingsvereistes tot tersiëre onderwysinrigtings. Pretoria: Raad vir Geesteswetenskaplike Navorsing.

I BUILT MY SITE FOR FREE USING