Psychological Constructs vs Attainment
The Role of Nonverbal Ability Tests in Identifying Academically Gifted Students: An Aptitude Perspective David F. Lohman. The University of Iowa. Gifted Child Quarterly (in press)
Abstract
The first goal of this article is to discuss the role of nonverbal ability tests in the identification of academically gifted children. I note that most nonverbal tests measure verbally mediated cognitive processes, that they are neither “culture free” nor “culture fair,” and that we have known these facts for a very long time. I show that selecting students for gifted and talented programs on the basis of such tests would exclude the majority of the most academically talented students in all ethnic groups. The second goal is to propose a better method for identifying gifted students. I argue that the critical issue is readiness for a particular type of educational opportunity.
The cognitive aspects of readiness are evidenced first in students. levels of knowledge and skill in particular domains and secondarily in their abilities to reason in the symbol systems used to communicate new knowledge. This applies both to minority and majority students. Therefore, the most academically talented minority students are those who show the strongest current achievement in particular domains and the best ability to reason in the symbol systems required for the acquisition of new knowledge in those domains. I also argue that although current accomplishment can be measured on a common scale, judgments about potential must always be made relative to circumstances.
I first learned about nonverbal ability tests in the early 1970s when I was taught how to administer many of these tests to the hearing-impaired students at the school for the deaf where I was working. By the mid-1970s I was in graduate school working on a research project that aimed to understand the cognitive processes people used when attempting to solve ability tests of all sorts. Because figural tests are particularly amenable to such inquiry, much of our work centered on these tasks (Snow & Lohman, 1984, 1989). Over the years, I conducted many studies on spatial abilities, David F. Lohman, College of Education. I am grateful to David Lubinski, Nick Colangelo, Susan Assouline, Katrina Korb, and five anonymous reviewers for their detailed and helpful comments on earlier drafts of this manuscript. I also thank Patricia Martin for expert assistance in preparing the manuscript.
Figural reasoning abilities, and on the nature of individual differences in thinking, problem solving, and their implications for instruction. In the early 1990s, I was asked if I would assume responsibility for the sixth edition of the Cognitive Abilities Test (CogAT; Lohman & Hagen, 2001a). As one who has spent 30 years studying figural reasoning and as the author of an ability test that has an excellent nonverbal battery, one might expect that I would be pleased with the recent emphasis on using figural reasoning tests to identify students for inclusion in programs for the gifted and talented. On the contrary, I am dismayed by the claims that have been made for such tests.
I see well-intentioned educators being misled about what these tests measure and, more importantly, children being hurt by selection policies that use nonverbal reasoning tests as the criteria of first resort “rather than of last resort” for admission to programs for the academically gifted and talented. The goals for using figural reasoning tests when selecting students for special programs for the gifted and talented are laudable: Measure abilities in a way that is fair to all students; increase the diversity of students who are included in programs for the gifted and talented; actively assist those who have not had the advantages of wealth or an immersion from birth in the English language. I endorse these goals. I also believe that figural reasoning tests can provide information that assists in achieving them. Such tests have a place at the selection table. But I disagree with those who claim that they should be at the head of the table or, worse yet, occupy the only chair at the table.
Contributors to this journal have disagreed on the proper role of nonverbal ability tests in the identification of academically gifted students. For example, Naglieri and Ford (2003) advocated the use of Naglieri’s (1997) group-administered figural reasoning test for identifying academically gifted students. Mills and Tissot (1995), on the other hand, counseled caution. They noted that large differences in the mean scores of ethnic groups and low correlations between scores on the Advanced Progressive Matrices Test (APM; Raven, Court, & Raven, 1983) and measures of achievement make it a poor primary selection tool for special programs that involve advanced coursework. As they put it: “Identification instruments should match the programs for which students are being identified” (p. 216). In their view, a more appropriate use of the APM may be as a screening test that, along with other assessments, could be used to identify academic potential in students who are not yet ready for advanced-level academic programs. Such students could be provided educational opportunities that aimed to develop academic skills needed to participate in advanced-level coursework. For reasons that I will explain later, I concur with their conclusions, although I also would argue that measures of quantitative and verbal reasoning should generally be considered before the nonverbal-reasoning test in the identification process.
Like Richert (2003), I also argue that rank within group on the most relevant aptitudes should guide efforts to identify academically promising minority students who are not yet ready for advanced-level academic programs. Overview In this article, I discuss the role of nonverbal ability tests in the identification of academically gifted children. I first give a brief overview of different types of nonverbal ability tests. These include a broad range of group and individual tests that measure an equally broad range of abilities. A major point is that to call a test nonverbal is to make a statement about the observable characteristics of the items that are presented and the responses that are required. It is not “or at least should not be” a claim about the cognitive processes examinees use to solve items.
I then discuss the assertion that such tests are “culture free” or “culture fair”. I argue that such claims mislead because they encourage the mistaken belief that abilities can be measured in ways that are independent of culture, experience, and motivation. This is not possible, and we have known it for a very long time. I then focus on nonverbal reasoning tests such as the Raven Progressive Matrices that require examinees to reason with figural stimuli. I argue that figural reasoning tests should not be the primary instrument for identifying academically gifted students. I first show that selecting students for gifted and talented programs on the basis of such tests would exclude most of the students who would profit from advanced instruction and include many who would not profit from it.
Understanding why this is so requires a grasp of the nature of human abilities, how they develop, and how they function as aptitudes for future learning. I begin by noting the correspondence between physical and mental abilities. This helps make clear the claim that all abilities are developed and that all tests measure developed abilities. I then discuss why it is important to distinguish among the abilities to reason with verbal, quantitative, and spatial concepts. I conclude this section by showing that a relative strength in spatial abilities seems actually to be an inaptitude for aspects of academic learning.
I next turn to the larger question of how we might best identify those students who either presently (or sometime in the future) would most profit from advanced instruction. I argue that the primary question to be addressed in selecting students for special programs is one of readiness for a particular type of educational opportunity, not innate ability. Readiness has cognitive, affective, and conative dimensions. I show that the cognitive aspects of readiness are evidenced first and foremost in students. levels of knowledge and skill in particular domains and secondarily by their abilities to reason in the symbol systems used to communicate new knowledge in those domains. Figural reasoning tests are generally distal predictors of readiness for academic learning.
Importantly, the predictors of current and future academic excellence are the same for minority and majority students. This means that the most academically talented minority students are those who show the strongest current achievement in particular domains and the best ability to reason in the symbol systems required for the acquisition of new knowledge in those domains. Finally, I argue that programs for the academically gifted should distinguish between high levels of current accomplishment in a domain and lesser levels of current accomplishment but potential for higher levels of future accomplishment. Acceleration to more advanced classes or instruction at an advanced level is often warranted for the high-accomplishment group, whereas intensive instruction at level or somewhat above that received by age-peers is often more appropriate for the high potential group.
Because judgments about potential are much more probabilistic than judgments about accomplishment, fixed cut scores for identifying high-potential students are difficult to defend, especially when students come from markedly different backgrounds. Two caveats at the outset: First, giftedness has many manifestations. Here I discuss only academic giftedness. This is not a judgment about the importance of musical or athletic or other types of giftedness, but a necessary concession to the limitations of space. Second, although identification of academically gifted students should include many sources of information (Assouline, 2003; Hagen, 1980), I focus on the role of ability tests, especially nonverbal ability tests.in the process.
Nonverbal Ability Tests
General Characteristics of Nonverbal Tests
Tests are commonly called nonverbal if items present visual stimuli such as concrete objects or line drawings and require a nonverbal response such as assembling a puzzle, pointing to an answer, or filling in a circle under a picture. Directions may be given verbally, in pantomime, or through feedback on the correctness of responses given to a set of practice items. Verbal directions are more common on group administered tests, whereas pantomimed directions or
Nonverbal Ability Tests and Identifying Gifted
Modelling of the desired behaviour are more common on individually administered tests. Verbal processes in nonverbal tests. To call a test nonverbal is to make a statement about the test stimuli, not the cognitive processes examinees use to solve test items. Indeed, “nonverbal” items commonly either require verbal or mathematical knowledge or use tasks whose solution is greatly facilitated by the use of verbal or mathematical cognitive processes. In some cases, these requirements are explicit. In some, they are less obvious.
An example of the explicit involvement of verbal processes is provided by the Pictorial Categories subtest of the Comprehensive Test of Nonverbal Intelligence (Hammill, Pearson, & Wiederholt, 1996). Each item on this subtest shows pictures of two objects at the top of the page and an empty box beneath them. The examinee must point to the object at that bottom of the page that goes in the box. The two pictures may be of an apple and a banana. The correct answer is another fruit rather than, say, a vegetable. A similar item in a verbal format would present the names of the objects rather than line drawings of them. Verbal analogies can also be presented in pictures, as on the Analogic Reasoning subtest of the Universal Nonverbal Intelligence Test (Bracken & McCallum, 1998). For example, some items present line drawings of analogies such as pear is to apple as carrot is to a) grapes, b) squash, c) tomato, or d) radish.
The major difference between the verbal and nonverbal format for such items is that the examinee is expected to comprehend spoken or written words in the verbal test and to decipher the line drawings in the nonverbal test. The pictorial format has both advantages and disadvantages. On the one hand, although the child must know words for both the objects depicted and the categories to which each belongs, that knowledge can be in any language. This is one of the main reasons why such tests are helpful when testing students with limited proficiency in English, especially when proficiency is so limited that the student cannot understand orally presented test directions. On the other hand, it can be difficult for the examinee to decipher a line drawing (e.g., Is it an egg or a lemon?).
A good example of the disconnect between stimuli and how they are processed is provided by a test of spatial ability that I devised to help select air traffic controllers. I called it the Verbal Test of Spatial Ability. Items were presented verbally (e.g., “Imagine that you are walking north. You turn right at the first corner, walk one block, and then turn left. In what direction are you facing?”), and required a verbal response (“North”). However, the test was one of the best measures of spatial ability in the selection battery (Ackerman & Kanfer, 1993).
Further, there are important regional and cultural differences in the familiarity of different objects and conventions in how they are depicted. For example, in the example above, there are regional and cultural differences in the vegetables children are most likely to see used at home. Particular items can also be harder or easier for students who speak different languages because of unexpected associations among words used to label the pictures.
On other tests, particularly those that use figural stimuli such as geometric shapes, the involvement of verbal processes is less obvious but equally important. The Figure Analogies subtest on the CogAT and the matrix completion items on the Raven Progressive Matrices and the NNAT are good examples of these types of tasks. Careful analyses of how examinees solve items on these tests (e.g., Bethell-Fox, Lohman, & Snow, 1984; Carpenter, Just, & Shell, 1990; Mullholand, Pellegrino, & Glaser, 1980; Sternberg, 1982) show that labelling of stimuli (e.g., as a square or diamond), of the attributes of that stimulus (e.g., large, shaded with vertical lines), and of the rule that will generate the missing elements (the figures get bigger and the background is combined) are critical for successful solution of all but the simplest items. Failure to label figures, their attributes, or the transformations that are performed on them substantially increases the working memory burden on the examinee. Since the largest source of individual differences on reasoning tests is working memory burden (Kyllonen & Christal, 1990), anything that increases this burden can significantly impair test performance. Indeed, the major source of error on figural reasoning tasks is forgetting a transformation, attribute, or element (Lohman, 2000).
Understanding what to do. The brief directions that are used on many nonverbal tests can create problems. For example, the most common mistake children make on analogy items is to pick a response alternative that is merely an associate of the last term in the stem (Achenbach, 1970; Sternberg & Nigro, 1980).
They are more likely to do this if they do not really understand how to solve analogies. Cryptic or pantomime directions do not instruct them in the process of saying to themselves: “Pear is to apple as carrot is to what?” and then checking that the relationship they have inferred between the first two terms can indeed be mapped on to the second pair of terms. The problem of not really understanding the directions goes beyond the analogy format. One study of the Raven Matrices showed that many minority children did not understand the directions. Going through the directions twice dramatically improved the scores of many of these students (Scarr, 1981).
Therefore, eliminating written or spoken words does not somehow render problems the same for all. Nor does reducing or eliminating verbal directions somehow level the playing field. In fact, it more commonly raises the slope for some.2 Confounding reasoning and spatial abilities. Finally, some figural tests measure spatial abilities, either inadvertently or explicitly. Most good tests of spatial ability require the examinee to perform an analog transformation on a mental image, such as mentally turning or rotating it to a new position.
Therefore, figural reasoning tests that require these sorts of processes are particularly likely to measure spatial abilities as well as reasoning abilities. Unless one intends to measure spatial abilities, this is not a good thing. The presence of sex differences provides a good way to distinguish between figural tests that measure spatial ability and those that measure reasoning abilities with figural stimuli. Good tests of spatial ability will show effect sizes for sex of .5 SD or more. Good nonverbal reasoning tests show no sex differences.
