Tuesday, April 14, 2009

The Bias of the SAT

"The overall SAT results were broken into 10 family-income blocks, beginning at less than $10,000. They increase in $10,000 increments to students with family income levels greater than $100,000. Students from families with less than a $10,000 income scored a mean of 429 in critical reading, which improved to 445 in the $10,000 to $20,000 income range. That score jumped in each of the next eight income groups, peaking at 549 with students from families earning more than $100,000. The same trend occurred in math: Students at the lowest-end income level had a mean score of 457, which crept to 465, 474, 488, 501 and then 509 in the $50,000-$60,000 range. The numbers kept improving to a mean score of 564 at the $100,000 and above level.
Typically, each $10,000 income increase corresponded to a 10- to 12-point gain in the mean score of each test section. The only significant variation was between students from families earning between $80,000 and $100,000 and those earning more than $100,000. In those categories, mean scores jumped 26, 30 and 29 points, respectively, in critical reading, math and writing.


Those results correlated with the Baltimore-area school district median income numbers. Howard County, with the United States’ third-highest median income, at $91,184, scored higher than Anne Arundel with median income of $71,961, which scored higher than Harford at $65,343. The trend continued down to Baltimore County at $56,295, which scored higher than Baltimore City at $32,456." - http://www.examiner.com/a-254205~SAT_scores_tied_to_income_level_locally__nationally.html?cid=rss-Baltimore






The graph is fairly straightforward and goes represents the core issue, that is that a particular test taker's household income has more predictive value than does his race. The question then becomes what exactly should be done to recognize this phenomenon and correct for it.

A logical starting question is whether or not race and household income (poverty in particular) are so intertwined as to undermine the value of making any distinction in the first place. According to 2007 census numbers approximately 10% of black households earn more than $100,00 per year. Meanwhile, that number is 21.5% for whites, 31% for Asians, and 11% for hispanics. My takeaway from this is that any racially based method of adjusting SAT scores would at the same time be over and under inclusive. It would doubtless provide a great benefit to a large number of minorities, whom are in a very real sense undervalued by such an income sensitive test. However, such a proposal would unjustifiably benefit tens of thousands of minorities that come from households earning more than $100,000. At the same time, a race based adjustment (whether directly or indirectly) would fail to provide a similar benefit to the hundreds of thousands of white and asian students coming from impoverished households.

I briefly take this time to point out that this analysis is equally applicable for most race based programs designed to compensate for the underrepresentation of minorities in colleges. Most of the justifications for such programs are fundamentally flawed. These arguments in support of race based programs mistake correlation for causation. The important reality is not that minorities are underrepresented because they are minorities, but rather minorities are represented because a higher percentage of them come from lower-income households. Therefore, any race based program is inherently not narrowly tailored to achieve an otherwise legitimate and praiseworthy goal.

Without a feasible way to adjust SAT scores for household income (in an imperfect attempt to account for the income bias) I might support the movement of some schools that have moved towards making the SAT optional.

(I disclose that the data does indicate race has some predictive value of SAT score performance apart from household income, but this predictive value is of a much smaller magnitude than income).

No comments:

Post a Comment