HOW TO MEASURE MERIT Those widely hated SATs turn out to be enormously useful in deciding who will do well at elite colleges.
(FORTUNE Magazine) – America's founding fathers believed that the new republic, in forsaking a hereditary nobility, must look to a ''natural aristocracy'' for its leadership. Two centuries later, the country generally seems to accept that principle: it agrees that those with the most ability, intelligence, and drive should have the top jobs and leadership roles and, in effect, are entitled to a kind of elite status. Not surprisingly, Americans attach enormous importance to the process by which some young people and not others gain this status. That process is the subject of Robert Klitgaard's Choosing Elites ($19.95, Basic Books), a fascinating inside look at some of the monitors of our meritocracy. Klitgaard is associate professor of public policy at Harvard's John F. Kennedy School of Government and a former chairman of the Admissions Committee of that school's Public Policy Program. For the past six years, he has also been an assistant to Harvard President Derek Bok, and in some measure Harvard's procedures serve as his model of how elites are chosen. Most people think of these procedures only in relation to the daunting difficulties they present for applicants. But in the first few sentences of the book, Klitgaard asks you to consider the flip side of the process: to look at the university's own difficulties as it tries ''to decide which among many superbly qualified applicants should be accepted.'' The question, he adds, is an ''archetypal instance'' of a general problem: ''If a few highly valued positions or opportunities are to be allocated among a large number of aspirants, how should this be done? What objectives should be sought? . . . What if certain subgroups are disproportionately excluded by selection according to individualistic criteria?'' Much of Choosing Elites is given over to analyzing what we do and don't know about the predictors of later-life success. Along the way, Klitgaard emerges as a powerful defender of the standardized ability tests -- the best known being the Scholastic Aptitude Test -- that have grown increasingly controversial. The tests are often criticized as unfair to disadvantaged students, but the criticism ultimately makes little sense. The SAT was developed because college admissions directors had so much trouble determining the real abilities of applicants from high schools of varying quality. If the students were admitted strictly on their command of mathematics or Latin or literature, then colleges would just be compounding the luck or privileges of those who had gone to St. Paul's or Bronx Science. A similar rationale led to aptitude tests for professional and graduate schools, such as the Law School Admissions Test (LSAT). How well do the tests work? Critics often say not well at all. Grades in college, they note -- correctly -- are almost as powerfully correlated with high school grades alone as they are with an index combining these grades and aptitude scores. High school grades and aptitude scores are themselves positively correlated, so that each echoes the other to some extent. Somewhat similar statements could be made about grades and test scores among applicants for the professional schools. All of which leaves critics concluding that we could drop the tests, and eliminate the financial and emotional costs that go with them, while sacrificing little in the way of predictive power. Klitgaard demonstrates, however, that this analysis isn't applicable to test scores at the ''right tail.'' That term refers to the extra-high scores in the lower right-hand corner of the bell-shaped distribution curve, which is where you find the candidates for elite status. In this universe, even a small increase in predictive power matters a lot. For example, a highly selective law school -- one that admitted only one applicant in ten, say -- could normally expect to reduce by two-thirds the already small proportion who fail if it considered LSAT scores in addition to college grades. THE FACT IS that test scores help a lot in predicting performance at the high (and also low) extremes of a population. Klitgaard cites, for example, a highly suggestive study of Harvard undergraduates in a large introductory economics course. The data indicate that a 25-point advantage in SAT scores helped a student master the course material as much as an additional ten hours of study per week. Another suggestive detail: the amount of material mastered under the best and worst instructors varied less than the material mastered by students with a 100-point difference in SAT scores. As everybody knows, admissions at most elite colleges are affected by a lot more than test scores and grades. The admissions committees keep close track of the impact on the alumni, on major athletic teams, on the student orchestra, etc. The committees also like to achieve some balance racially and ethnically, believing that balance is required by considerations of fairness and diversity, not to mention the law. However, they have a persistent problem: applicants from minority groups, particularly blacks and Hispanics, have lower average scores on the SATs and other aptitude tests than do white and Asian-American applicants. This problem is greater at elite institutions than at others because the group differences are amplified to an extraordinary degree at the right tail. Minority representation would be far lower than it is at elite institutions if their admissions committees were color-blind in looking at test scores. According to Klitgaard's estimates, the minority representation in law and medical schools would tumble dramatically (by as much as 90% in some cases) if admissions committees looked at everyone's test scores evenhandedly. At one (unnamed) elite undergraduate college for which he had data, Klitgaard tells us that the drop in minority representation would have been about 85%. A common reason for not looking at test scores evenhandedly is that the tests are said to be biased against applicants from minority groups. If they truly were biased, you would expect the tests to underpredict subsequent academic performance -- that is, minority students would be getting higher grades relative to test scores than would white students. But the charge of bias doesn't hold up, at least not with respect to black and Hispanic students. Klitgaard notes that the SAT and other aptitude tests have been found ! repeatedly to overpredict academic performance for most groups targeted for affirmative action. The general rule is that when white students and minority students (excluding those of Asian heritage) have identical test scores, the white students will do better as undergraduates; they will also do better in the professional schools. Test bias, in the sense that the term is understood to mean underpredicting performance, is mainly a problem for whites. Arguments about admissions standards tend to turn into arguments about competing values, and nowhere more than in the standards applicable to minority groups. How do administrators respond to demands for ''equity in admissions'' -- meaning some kind of substantial representation for each group -- when the different groups reliably generate different average scores? In this situation, equity in admissions implies a performance gap that inevitably stigmatizes members of the groups favored by the admissions process. The pressures for equal representation and those for equal performance are a genuine and inescapable dilemma of competing values. The dilemma is hardest to deal with at the right tail, where Klitgaard shows the performance gap to be greatest. Even without the thorny issues raised by minority representation, selecting an elite is an uncomfortable business. Individuals vary enough so that selection by lottery makes no sense; yet selection by merit-based criteria requires decision-makers to be clearer than most of them are about the attributes they're looking for. A major virtue of Choosing Elites is that it can help them see more clearly the consequences of alternative policies. |
|