Cross cultural investigation

THE RELIABILITY AND VALIDITY OF THE PIB

Pieter Schaap, D.com

(Department of Human Resource Management) (University of Pretoria)

APPROXIMATIONS OF THE CULTURAL GROUPING ON WHICH THE DATA IS BASED

Cultural groupings	Percent
Black people	60-70
White people	10-15
Asian people	15-20
Coloured people	5-10

The data is more or less in line with the demographics of the RSA

SUMMARY OF PIB RELIABILITY STUDIES DONE SINCE 1995

		Reliability Studies
PIB-Index (Verbal)	Total	Study 1		Study 2		Study 3		Study 4		Study 5		Study 6		Mean
	N	N	Rtt	N	Rtt	N	Rtt	N	Rtt	N	Rtt	N	Rtt	Rtt
1-General Knowledge	2295	526	0.70	106	0.74	1663	0.71
2- Creativity	8086	1666	0.71	101	0.72	1794	0.78	4525	0.75	3038	0.75
3-Reading Comprehension	35964	3782	0.79	86	0.65	11454	0.83	7450	0.83	13192	0.76
4-Calculations	394	291	0.74	103	0.89									0.81
5-Mental Alertness	36802	3819	0.87	1167	0.93	11108	0.87	7240	0.87	13468	0.79			0.87
6-Listening Skills	335	335	0.70											0.70
7-Abstract Reasoning	2067	236	0.77	1831	0.76									0.77
8-Flexibility	3748	2195	0.79	1553	0.70									0.75
9-Interpersonal Relations	14304	187	0.83	1148	0.79	2694	0.83	5102	0.82	2978	0.82	2195	0.85	0.82
10-Self-image	7242	736	0.76	2374	0.87	2749	0.79	1567	0.79					0.80
11-Vocabulary	16790	290	0.76	1803	0.76	1340	0.76	13357	0.76					0.76
15-Motivation	9305	1163	0.70	1422	0.75	2687	0.71	1930	0.71	2103	0.82			0.75
17-Time management	587	587	0.70											0.70
18-Stress Level	5762	1948	0.94	1585	0.92	2229	0.90							0.92
20-A\B Behaviour	186	186	0.70											0.70
21-Assertiveness	6736	187	0.88	148	0.72	2241	0.83	1950	0.74	2210	0.86			0.80
22-Spelling	14034	566	0.65	13468	0.74									0.70
24 Coping Skills	187	187	0.61											0.61

Reliability Studies

PIB-Index (Verbal)

Total

Study 1

Study 2

Study 3

Study 4

Study 5

Study 6

Mean

Rtt

Rtt

1-General Knowledge

7389

703

0.80

1564

0.79

4564

0.79

558

0.85

0.81

2- Creativity

701

0.61

3-Reading Comprehension

700

700

0.80

0.80

4-Calculations

701

0.68

5-Mental Alertness

5317

1350

0.84

3353

0.84

558

0.89

356

0.89

0.86

6-Listening Skills

356

356

0.78

0.78

7-Abstract Reasoning

356

356

0.89

0.89

Abbreviations:

Rtt: Reliability

N: Sample size

SUMMARY OF PREDICTIVE VALIDITY STUDIES DONE SINCE 1995

The following results is based on regression analyses done and includes more than one predictor (PIBindexes):

Mean R

Academic environment (based on 42 studies) 0.70

Corporate environment (based on 4 studies) 0.40

The following PIB-Indexes have shown predictive validity in a single or more than one study.

Verbal Indexes:

1- General Knowledge

2- Creativity

3- Reading Comprehension

4- Numerical Ability

5- Mental Alertness

7- Abstract Reasoning

8- Flexibility

9- Interpersonal Relations

10- Self-image

12- Vocabulary

15- Self-motivation

17- Time Management

18- Stress Management

21- Assertiveness

22- Spelling

Visual:

2- Spatial Reasoning

3- Short-term Memory

4- Numerical Ability

5- Mental Alertness

6- Potential to Assemble

SUMMARY OF BIAS ANALYSIS DONE SINCE 1995

The following indexes were analyses to date and found to be non-biased for cultural and language groups in RSA:

1- General Knowledge

3- Reading Comprehension

5- Mental Alertness

12-Vocabulary

22-Spelling

SUMMARY OF CONTRUCT VALIDITY DONE SINCE 1995

A. INTER-TEST CORRELATION STUDIES

PIB-Indexes (Verbal)	INTER-TEST CORRELATIONS (N=200 or more)
	SSHA	AATnv	AATv	AATs	VPib-4	FCT	16PFc	Lead	Anx	Ext
4-Calculations						0.53
5-Mental Alertness	0.58	0.70	0.45	0.52	0.59
9-Interpersonal Relations	0.58						0.45	0.63	-0.43	0.51
10-Self-image							0.70	0.55	-0.51
15-Motivation	0.62						0.53	0.33	-0.52
18-Stress Level	-0.54
24-Coping Skills	0.69

Abbreviations:

SSHA: Survey of Study Habits and Attitudes

AATnv: Academic aptitude test (non-verbal)

AATv: Academic aptitude test (verbal)

AATs: Academic aptitude test (spacial)

VPIB-4: Calculations test

FCT: Figure classification test

16PFc: 16PF factor c (emotional stability)

Lead: Combination of 16PF factors closely associated with leadership

Anx: 16PF anxiety (second order factor)

Ext: 16PF extroversion (second order factor)

FACTOR VALIDITY AND ITEM ANALYSIS STUDIES OF SELECTED PIB INDEXES

Procedures followed:

In analysing the indexes the following procedure was used in the case of fixed interval scaled items (Schepers,1992). A first-order factor analysis was performed using the principle axes factoring method and varimax rotation. Items with high loadings on a specific factor were added together to form a sub-value. A second order factor analysis was performed on the sub-values using the principle axes factoring method and an oblique rotation.

The index scales were identified according to the factors extracted on the second order analysis. Items were grouped according to the loadings on a specific factor. Item and reliability analysis were done separately for each scale Item selection was based on item-total correlation, item mean values and item variance. In the case of the dichotomous scaled items the following procedure was used: An item analysis was done and the best items were selected based on the difficulty value, discrimination value and the item-total correlation.

Reference:

Schepers, J.M., 1992. Toetskonstruksie: Teorie en praktyk. Johannesburg: RAU-Drukpers Updated and new PIB-scales: N Rtt (reliability) Index 1: General Knowledge (20 items) 1663 0.71

Index 2: Creativity 1727

Scale 1: Rigidity (15 items) 0.72

Scale 2: Imagination (15 items) 0.83

Index 3: Reading comprehension (25 items) 13192 0.76

Index 4: Calculations (15 items) 615 0.72

Index 5: Mental alertness (25 items) 13468 0.80

Index 7: Abstract reasoning (18 items) 1831 0.76

Index 8: Flexibility 2195 0.79

Index 9: Interpersonal relations 2828

Scale 1: Sociability (19 items) 0.80

Scale 2: Associability (18 items) 0.80

Index 10: Self-image 2421

Scale 1: Self-worth (21 items) 0.82

Scale 2: Self-critism (15 items) 0.81

Index 12: Vocabulary (20 items) 13192 0.76

Index 15: Motivation 1436

Scale 1: Internal locus of control (20 items) 0.81

Scale 2: External locus of control (20 items) 0.81

Index 18: Stress level 1984

Scale 1: Physical symptoms (12 items) 0.81

Scale 2: Psychological symptoms (24 items) 0.94

Index 21: Assertiveness (25 items) 2241 0.83

Index 22: Spelling (16 items) 13468 0.74