We can’t decide how we feel about standardized college admission tests like the ACT. Some schools think they’re fundamentally biased against female, minority and low-income students and have either stopped using them or made them optional. Others believe that abandoning the tests will harm low- and middle-income students by making the admission process more subjective, thereby advantaging wealthier, better connected applicants. On the other hand, some schools think that standardized tests are necessary to accurately assess student preparedness and identify those who will benefit most from college. Such schools have either continued or reinstated their test requirements.
Is there anything more to add to the debate, or are the tests just fatally flawed anachronisms?
Most of what I’ve read about standardized tests like the ACT has been written from the “30,000-foot perspective” by people who have faced the test once as teenagers. I’d like to offer a different perspective. I’m a biological psychologist. I’ve been a university professor in both Psychology and Biology departments for several decades. However, I also spent 12 years teaching high school biology, chemistry, math, and ACT/SAT prep courses for a nonprofit tutoring center which, unfortunately, recently closed. They did great work.
The Center (as I will call it) was unique in that it only hired experienced, professional teachers. It drew students from about a dozen area high schools ranging from large, diverse, urban public schools to small, very elite private prep schools. The students were as diverse as the schools from which they came. Some had unique learning challenges, some were high achievers intent on honing their skills. Most were in between.
I didn’t do typical ad-hoc tutoring at the Center. Students who needed help met with me individually, one hour every week for the whole semester. I mentored them, lesson-by-lesson, according to their individual needs. Some worked with me over their entire high-school careers, sometimes I tutored their siblings.
These individualized, weekly sessions exposed me to the lesson plans, textbooks, and expectations from a variety of area high schools. In addition, because I was also teaching ACT prep courses, I learned how well the ACT reflected what my students were being taught in school. I wouldn’t have gained that perspective as a typical classroom teacher.
Perhaps this close match between the ACT and the high school curricula is why the test is a better predictor of first-year college success than are grade point averages (which are subject to grade inflation).
The Center taught test prep using actual ACT tests. In order to stay abreast of periodic updates to the test, and to understand how my students experienced each test, I took all of the ACTs archived at the Center and each new version as it was released. I completed each under test conditions, timed and with no answer key. I’ve taken at least 20 ACTs. I never got a perfect score, but I understand the test better than most.
Do recent ACT results suggest bias?
In 2021, approximately 1.3 million students took the ACT. The test has four sections—English, Math, Reading, Science—with a total of 215 multiple-choice questions. There is also an optional writing component. The number of correct answers for each multiple-choice section is converted to a scaled score ranging from one to 36. The overall composite score is the average of the four scaled scores, rounded to the nearest whole number. Thirty-six is a perfect score.
Last year, female students earned a slightly higher average composite score (20.6) than males (20.3), and outscored males in both the English and Reading sections. Females were virtually tied with males in Science (20.4 and 20.6, respectively), and differed by less than a point in Math.
Among those students who took the most rigorous series of high school science courses (General Science/Earth Science, Biology, Chemistry, and Physics), there was less than one point difference in the average Science scores between males and females (23.4 versus 22.5).
Females also outscored males in the Optional Writing Test (19.6 versus 18.1). In addition, a greater percentage of females met the College Readiness Benchmark Scores for English and Reading irrespective of the particular mix of courses that each group took in high school.
These data do not support the claim that the ACT discriminates against females. How, then, might we think about other claims of bias?
Students who take the ACT are asked to racially self-identify (not all do). These are the average composite scores for the five largest racial categories, followed (in parentheses) by the composite scores for those students who took four or more years of English and three or more years each of math, social studies, and natural sciences: Black = 16.3 (17.9), Hispanic/Latino = 18.3 (20.3), White = 21.7 (23.3), Asian = 24.9 (26.7), Two or More Races = 20.6 (22.5), and No Response = 19.2 (23.7). Clearly, taking a more rigorous high school curriculum helps everyone.
To put these scores in perspective, if we assume that the correct multiple-choice answers are randomized across all questions, and the test taker simply chooses the answer “C” for every question, her composite score would be 12–13. This would vary only slightly from test to test. A score of 20 marks the 50th percentile (half of the students scored below 20).
A scaled score of 23 on each of the four sections (yielding a composite score of 23) would meet each of the ACT College Readiness Benchmarks. You’ll notice that at least one of the composite scores in five of the six racial categories are below 23, meaning that at least one benchmark was not met irrespective of racial category.
These composite scores are the ones used to support the claim that the ACT is racially biased. However, if the scores are considered in terms of self-identified race and self-reported postsecondary aspirations, a more complex picture emerges.
For instance, black students who aspire to graduate study or a professional degree had composite scores of 19.5 and 19.6, respectively. These were higher than Asian or white students, whose aspirations were to pursue a vocational/technical degree or a two-year college degree (16.9 to 18.5). Further, the average scores for black students aspiring to a graduate/professional degree differed from white students who aspired to a bachelor’s degree by only 1.3 points.
Even more telling is the fact that within all self-identified groups, composite scores increased as aspirations increased from vocational/technical training to graduate study and professional level degree. The largest increase was a remarkable 52 percent among Native Hawaiian/Pacific Islander and Asian students. Hispanic and black student scores increased 46 percent and 40 percent, respectively, between those seeking a vocational versus a professional degree. The smallest increase, 27 percent, was among white students.
There were also aspirational differences within planned educational majors between students who were planning to attend two years of college and those planning to attend four years. The latter group had consistently higher composite scores irrespective of anticipated college major. The biggest differences, 7 to 8.7 points, were among students planning to major in Engineering, English & Foreign Languages, Ethnic & Multidisciplinary Studies, and science (Biological & Physical, Computer Sciences & Math). The smallest differences (1.9 to 2.7 points), and the lowest overall composite scores (16 to 18.7) were among students planning to major in Community, Family, & Personal Services, Health Administration & Assisting, and Repair, Production, & Construction.
These data indicate that something other than racial and ethnic differences are at play. Perhaps, it’s linked to whatever socio-economic, personal, cognitive, or behavioral traits influence one’s aspirations and career goals.
Differences between high- and low-performing students
This “experiment” was unintended and serendipitous. The Center regularly held six-week ACT prep courses at several high schools. The classes were voluntary, met for four hours on six consecutive Saturdays, and were paid for by the students’ parents. Students took a pre- and a post-test before and after the course, respectively.
One of the high schools that hosted the course was a large, public school serving about 4,000 students from seven surrounding suburbs. It’s still considered one of the best in the state and spends about $15,000 annually per student. Overall, the school’s average composite ACT score is 27–28. This is the kind of cohort that’s often used as an example of students with an unfair socioeconomic advantage in tests like the ACT. In the aggregate, it seems to be so. But, there’s more to the story.
During one semester, 39 students signed up for the course. To make the course as effective as possible, the students were divided into two groups, those who scored above and those who scored below 25 on the pre-test. The students were taught the same lessons by the same two teachers on the same days. I taught the Math and Science sections and a colleague taught English and Reading.
Although drawn from the same student population, and similar in demographics, the difference between the two classes was dramatic. Those in the over-25 group paid attention, raised their hands, asked and responded to questions, shared the solutions to difficult problems, spontaneously helped each other, and—most importantly—did the homework. Many in the under-25 group were rambunctious (to put it mildly). Others were uninterested. Many spent much of the time staring out the window, most could not respond to questions. Few did the homework.
All of these students were great people, and I thoroughly enjoyed working with them all. However, as a psychologist, I would surmise that the students in the two classes were aspirationally different. Whether that’s true or not, their behaviors and outcomes were certainly different. The classes began with significantly different average pre-test scores (19 and 28) and, despite the fact that they both improved a little, they ended up with significantly different post-test scores (22 and 29).
This suggests that the ACT does, in fact, discriminate but not in terms of race, gender, or socioeconomic status, per se. This classroom “experiment” suggests that students with better classroom behavior and study habits do better on the test. The 2021 national data suggest that students with more lofty aspirational goals do better than those with lower self-expectations irrespective of race. Of course, any teacher could’ve told you that.
I recognize the fact that there are a variety of other social, cultural, economic, cognitive, familial, and personality factors that influence both student aspirations and classroom performance, but these have nothing to do with the worth of the ACT as an indicator of how much a student has learned.
As a diagnostic tool, the ACT is invaluable in identifying students who most need help in preparing for college, a vocation, or the mundane tasks of adulthood (like managing a household budget). Just knowing that only 25 percent of the 1.3 million students who took the ACT in 2021 met all four College Readiness Benchmarks, and that 55 percent (711,705 students!) were “below proficient” in understanding complex texts, are critical pieces of information if one seriously wants to help students succeed.
Is money the barrier to a good ACT score?
One criticism of standardized tests like the ACT is that only the well-to-do can afford “expensive” prep courses like the ones that I taught. (The Center’s course cost a few hundred dollars.) Critics always stress “expensive” to emphasize their claim that prep courses give an unfair advantage to wealthier students.
I think this is wrongheaded for a couple of reasons. First, after a decade in the field, my honest opinion is that test prep courses are not that helpful (even the ones that I taught). Second, you can get all of the test prep help that you need online for free.
Test prep courses can be helpful in only two ways. First, an experienced teacher can familiarize students with the format of the actual ACT (which is rarely the case in for-profit prep courses). This reduces test anxiety which interferes with test performance. Second, a knowledgeable teacher can review key material as it’s taught in high school. (This, too, is rarely the case in for-profit prep courses.) This helps students remember the details of their past lessons and reminds them of what they need to review.
These are the only two things that a prep course can do. There are no “secret strategies” to taking the test. This misconception is driven by test publishers who claim that their books and courses provide “Powerful tactics to help you avoid traps and beat the ACT.” However, there are no “traps” or trick questions in the ACT, and it’s not an enemy that needs to be “beaten.” These silly claims fuel the erroneous belief that students who can’t afford a prep course are being excluded from some special inside information. They are not.
More to the point, test prep courses are available free in many high schools, and all of the resources necessary to practice the tests are available online. All you need is a smart phone (they have built-in calculators), a pencil, and paper. While there may still be social and circumstantial barriers to accessing these materials, wealth is not one of them.
It’s important to reiterate that if a student—for whatever reason—did not or could not learn the basic material taught in school, test prep courses will not help. A prep course cannot substitute for four years of high school. Consequently, accessibility to a prep course is not what’s standing between an unprepared student and college admission. It’s unfair to students to claim otherwise.
How much does a prep course help?
I analyzed the performance of 205 students who took the Center’s ACT prep course during one academic year. Students took the course either at their school or at the Center, itself. Some students chose not to take either the pre- or post-test which left 183 post-test scores and 170 matched pairs of pre- and post-test scores to analyze. I did a lot of fine-grained statistical analyses but the primary take-home message was straightforward.
The overall average composite pre-test score was 22.8. After completing the course, the overall composite score was 23.6. Although this is a mathematically significant increase, there is no real, practical difference. Further, even though the high schools from which the students came differed, there were no differences in student improvement between schools.
Of course, not all students did better on the post-test. Some did worse. However, when I considered just those students who improved (presumably the most motivated), the results were basically the same. The overall average gain was 2.8 points. The median improvement was two points. Again, this is mathematically significant but of modest practical import.
Interestingly, those who scored below 25 on the pre-test improved more than those who scored 25 or above (an average of three versus 1.5 points). However, again, the group that started out below 25 ended up with an average composite score significantly lower than students who scored above 25 on the pre-test (23.5 versus 29).
Beyond simply becoming familiar with the format of the test, I don’t think that a prep course makes a big difference. These data do not support the contention that the inability to take a prep course, per se, is preventing deserving students from getting a college education. A couple of points on the ACT will not change the trajectory of anyone’s academic career.
The elite college myth
One of the more ill-conceived justifications for eliminating standardized tests is that they erect unfair barriers that prevent poor, working class, and minority students from attending the most prestigious colleges and universities thereby limiting their potential for success. I think this is a ruse.
In the first place, the so-called elite schools admit only a very small portion of college students overall, and a similarly small portion of those who apply (irrespective of their standardized test scores). So, even if we added 10 points to every applicant’s ACT score, or eliminated the test altogether, the probability that any particular student would be admitted into an elite school is vanishingly small. Besides, even with acceptable credentials, not every student will thrive in an elite, Ivy League environment. Thomas Sowell has made this point repeatedly.
More to the point, going to an elite school is neither necessary nor sufficient for success. It takes only a few seconds on Google to find scores of successful people who went to non-elite or middle rung (or lower) colleges and universities. Consider these few: Former Disney CEO Michael Eisner went to Denison College; Former US Secretary of State Rex Tillerson went to University of Texas-Austin; Congresswoman Maxine Waters went to California State University-Los Angeles; Warren Buffett went to University of Nebraska-Lincoln; President Joe Biden went to the University of Delaware; Actress Halle Berry went to Cuyahoga Community College; Kevin Johnson and Howard Schultz, CEOs of Starbucks, went to New Mexico State-Las Cruces and Northern Michigan University, respectively; Tom Hanks went to Chabot Community College and eventually dropped out of California State University-Sacramento; and, Steve Jobs famously dropped out of Reed College in Portland, Oregon. Clearly, it’s not the schools that made these people successful.
Likewise, we all know people with remarkably good—sometimes elite—educations who are living modest, non-celebrity but very productive, happy, and meaningful lives. I’m one of them. Conversely, there are dozens of very good schools that admit students with modest ACT scores. Even an elite school like my alma mater, the University of Chicago, will sometimes admit students with ACT scores as low as 20. Standardized test scores are just one component of the overall admission decision.
There are many paths to a great education and a successful life. Going to an elite university is not necessarily one of them.
What I’ve learned
I’ve been teaching for quite a while. Over the decades, a lot has changed, but much has stayed the same. For instance, students’ intellectual needs haven’t changed very much. Nor has the basic level of education necessary for a reasonably good life. (No, everyone does not need to code.)
Something else has stayed the same. It’s the students’ biology. Learning is actually a physiological process. In order to learn something, there has to be an actual change in brain organization. That’s why learning is difficult, and why it takes time. It’s no different from sports. In order to get better at tennis, you have to practice tennis. In order to learn math, you have to practice math.
To do well on a test like the ACT, you have to take it after having thoroughly and repeatedly practiced the material that it covers. That takes several years of high school, just like it takes several years to be a good tennis player.
Does the ACT discriminate? Yes, it does. It separates students based on how much high school material they’ve learned. That’s all.
