Validity Study

1. Introduction

Technicon Pretoria started a potential assessment process on a large scale in the late nineteen nineties. In order to adhere to legal and professional guidelines for the use of psychometrics a continuous research project was launched in 1996. This project has as its aim the determining of the situation-specific validity and reliability of potential assessment tools and batteries.

The results of the aforementioned studies are continuously incorporated in prediction models used for selection purposes. This study is the first of its kind on SpEEx used for the admission of students to the Biotechnology and Food Technology courses presented at Technicon Pretoria.

2. Purpose of report

The purpose of this report is to establish the predictive validity of the potential assessment battery used for the selection of students for the courses in Biotechnology and Food Technology.

3. Methodology

For the purpose of this study a non-probability convenience sample was selected. The subjects were the total number of first-year students enrolled for the courses in Biotechnology and Food Technology in 2001, of whom both psychometric and academic data were available. Due to the fact that the classes were relatively small, the psychometric results for the two groups were combined and the academic subjects both groups took, focussed on as criterion variables.

The statistical analysis in this study involved the following steps: P The mean and standard deviation for all predictive variables were calculated. A frequency distribution for each variable was drawn. Attention was given to the median, as well as the skewness and kurtosis of each distribution. P A multiple regression analysis was performed with the students. psychometric results as predictor variables and academic performance as criterion variables.

4. Results

Table1 contains a summary of the descriptive statistics of the results obtained by the respondents on the indices included in the potential assessment battery for prospective students in the Biotechnology and Food Technology courses.

TABLE 1: Descriptive statistics of results obtained by the respondents on the indices of the potential assessment battery.

When interpreting the descriptive statistics, it is important to realise that these statistics describe the score obtained by a preselected group. Only the scores of those students who were finally admitted to either the Food Technology or Biotechnology course were included in the analysis. The frequency distribution of the results obtained by the respondents included in the sample are presented in Appendix A. It is inevitable that the distributions of scores will not be normal in all instances, due to the fact that a preselected group was used in the study. Cognisance of the fact that this might influence the validity of the prediction model was taken.

Table 2 gives the results for the multiple regression analysis performed on the data. The results of the various academic subjects were used as criterion for the analysis.

TABLE 2: Result of multiple regression analysis performed on data.

(n = 106)

From Table 2 the validity coefficient (R) for each criterion variable can be seen and the percentage (R2*2) of the variance in the criterion variable explained by the variance in the predictor variable included in the prediction model, can be deducted. Furthermore, the standard error of estimate indicates the accuracy of the prediction. The F Ratio indicates whether the relation between the actual and predicted scores are statistically significant.

5. Conclusions and Recommendations

5.1 The results indicate that the respondents. performance on the Environmental Exposure subtest, played a role in the prediction of their performance in various academic subjects. This might be explained by the fact that the test indicates the ability or capacity to expose oneself to a stimulating environment and to accumulate knowledge from the environment. This capacity is related to the ability to learn new facts, which in its turn leads to good performance in subjects with a factual content. Theorists generally believe that there is a positive relation between general knowledge and general intelligence.

5.2 The students. performance on the Advanced Calculations index also played an important role in the prediction of their academic performance. This index measures the capacity to work and deal with numbers and figures of advanced complexity.

5.3 The three indices Insight, Observance and Conceptualisation forms a unit, measuring the general cognitive and reasoning ability of the student. These indices are also associated with the general mental ability or G-factor of the testee (Schaap: 1997:70). All three of these indices played a role, to an extent in the prediction of the students’ academic performance. The role played by the Insight-index seemed to be somewhat more important though - an observation that might be explained by the fact that the Insight-index is the one of the three that asks of the testee to reason in language, whereas the other two indices make use of visual items.

The deduction could thus be that the ability to use language might be of utmost importance in the academic performance of students. The exact influence of language ability should therefore receive attention.

5.4 Because of the fact that the sample consisted of a preselected group, the restricted range effect on the validity coefficient should be taken cognisance of. Restriction in range occurs when the coefficient is lower than the actual relationship when the range of scores on one of the variables is confined to a representation of only a part of the total distribution of that variable (MacMillan & Schumacher, 1993:277). In this instance the range is restricted to students who have higher SpEEx scores.

5.5 As the sample of this study is relatively small, the results should not be generalised further than the specific situation it was calculated for. The findings should also be validated in future studies.

6. References

MacMillan, J.H. & Schumacher, S. 1993. Research in Education: A Conceptual Introduction. 3rd Edition. New York: Harper Collins College Publishers. Schaap, P. 1997. Guidelines for Evaluating and Validating Assessment Tools. In: Erasmus, P.F. (Ed) The Unfair Labour Practice the Worker and Assessment. Krugersdorp: RNATA.