Motivated Reasoning Is Disfiguring Social Science
The bottom line is that professional guilds such as the APA and AAP have a demonstrable track record of unreliability when speaking on matters of science.
On February 15, the American Psychological Association (APA) Council of Representatives voted for a resolution opposing parental spanking (full disclosure: I serve on the APA Council of Representatives but speak only for myself). The resolution statement presented spanking research as if data conclusively links spanking to negative outcomes in children such as aggression or reduced intellectual development. I happen to do some research on spanking’s effects on children. Although I am by no means a spanking advocate, I was alarmed by the way an inconsistent, correlational, and methodologically weak research field that routinely produces weak effect sizes was mischaracterized as consistent and strong. Unfortunately, this resolution is part of a larger bias among professional guilds such as the APA, wherein messy science is laundered for public consumption, presenting it as more impressive than it actually is.
Many people in the general public believe that organizations such as the APA or American Academy of Pediatrics (AAP) are neutral, objective scientific organizations or that they are even part of the government. But they are largely professional guilds wherein members like me pay dues to support their profession (both, but particularly the APA, are also publication houses). As a result, such organizations tend to market their fields much like any business markets its products. Impressive sounding science brings the field prestige, captures the attention of policy makers and helps members with grants, newspaper headlines, and career advancement. This isn’t necessarily a bad thing, and I certainly don’t wish to imply any bad faith. But failing to understand this basic fact may cause the general public to overestimate the degree of objectivity with which such organizations speak when talking about research.
There are good and honest arguments for why parents might choose other discipline strategies aside from spanking. Unfortunately, these were not arguments the APA made in the voted resolution. In a recent meta-analysis I conducted with several colleagues, we found that the effects of spanking on child aggression were so weak that they were best interpreted as negligible. Indeed, depending on how one looks at the data, it’s possible to make it look like spanking has either tiny negative or tiny positive impacts on children’s behavior.
Not all scholars agree with our assessment to be sure. But the argument that spanking correlates with child aggression (and the data is almost all correlational) mainly relies on effects that don’t control for other variables, such as the child’s pre-existing behavior problems (presumably misbehaving kids get spanked more) nor more serious forms of abuse. When studies do control for such factors, effect sizes become trivial, indeed about the same size as the impact of wearing eyeglasses on suicide. Naturally, we don’t warn parents about the public health risks of children’s eyeglasses because that would be silly. But the larger issue is not the tendency to pick sides in this debate, but that the APA is pretending that no debate exists at all.
Spanking is one of those ideologically and emotionally loaded cultural crusades that is ripe for the misuse of inconvenient data. Most of the members of the APA task force that produced the resolution are long-time anti-spanking advocates, reviewing their own research and declaring it beyond further debate. This has been a consistent problem for the APA and other professional guilds such as the AAP (more about which in a moment), and they have repeatedly allowed scholars with fairly straightforward conflicts of interest to produce resolutions that might reasonably be expected to confirm their ideological biases. This process is not remotely objective.
Ideologically loaded statements have become more common in recent years. The APA has had a particularly bad run of it. Just this past January, the APA’s practice guidelines for men and boys became a polarizing lightning rod, beloved by some cultural progressives, but largely panned in the centre and on the Right as stereotyping and arguably sexist in their portrayal of men and “traditional masculinity.” The guidelines were long on progressive terminology but relied on weak evidence and ignored entire fields of research related to the biological roots of gender. One may agree or disagree with the APA’s position on this matter, but if the guidelines were intended to help therapists understand men from backgrounds and perspectives different from their own, or to encourage men who were wary about therapy to give it a try, the guidelines clearly failed.
John Staddon
And then there is the long-running controversy over the APA’s 2015 resolution on violent video games. This happens to be the field I spend most of my time in, having provided testimony to former Vice President Biden’s hearings on the Sandy Hook shooting and testified before the School Safety Commission in June 2018. In 2013, over 230 scholars wrote to the APA expressing concern as the task force was just getting started. As with the other two resolutions, the task force appeared largely stacked with people who had taken prior anti-game views and portrayed a research field that simply doesn’t exist. Indeed, most scholars now agree that violent games play no role in societal violence. But the APA’s resolution has become a go-to source for politicians and the National Rifle Association whenever they want to distract the populace from talking about guns after a high-profile shooting.
Just this year a committee (which I chair) of the APA’s own Society for Media Psychology and Technology published a report documenting serious errors, biases, and misstatements in multiple resolution statements produced by the APA, AAP, and other professional groups when talking about media effects. In each case, the reports misrepresented the evidence for media effects, often simply ignoring any research that conflicted with the group’s stated position.
The bottom line is that professional guilds such as the APA and AAP have a demonstrable track record of unreliability when speaking on matters of science. This means that parents, the general public, and policy makers may base decisions on erroneous pseudo-scientific claims that can’t be backed by good data. Perhaps the most egregious issue is when such bodies simply pretend no controversy exists in fields that are, in fact, highly controversial. This behavior, known as “citation bias,” has been described by some scholars as one of the seven deadly sins of research scholarship. As professors, we would give a student a failing grade for such behavior in an academic paper. And yet professional guilds engage in such behavior on a fairly regular basis, at least with respect to behavioral research.
Why does this happen? There are a number of possible explanations.
The first is a lack of intellectual diversity. It’s been known for decades that the social sciences are heavily weighted with individuals who identify as sociocultural liberals or progressives. It probably isn’t just a happy coincidence that so much of social science is in lockstep with liberal and progressive social advocacy positions.
The second is the culture of institutions. From my experience and perspective, these tend to function on a corporate structure. Although they are typically non-profits, they increasingly behave like businesses rather than academic centers. As such, they do not appear to foster an appropriate level of critical thinking, skepticism, caution, or solicitation of opposing views (indeed on the spanking resolution, one skeptical scholar, Robert Larzelere, who volunteered to help was explicitly rebuffed). This is a recipe for conformity and groupthink. (Indeed, official APA policy appears to forbid scholarly special interest groups under its fold from taking public positions that differ from its own central stated positions…consistent with a business but not an academic or scholarly model.)
Third, and related, the review processes these resolution statements undergo is obviously failing. Often, on council, advocates for a position will boast that a resolution passed through multiple boards and committees, all internal to the APA. And yet, I find myself thinking, the resolution still sucks. Indeed, the “this has been reviewed so many times before” line of argument is simply an appeal to consensus and authority, both of which expose the failure of such organizations’ review process, not any superior quality of the resultant product.
Worryingly, professional guilds are producing resolution statements with increasing frequency. Indeed, our review of media effects policy statements found that the AAP, in particular, is producing them like a moral panic pez dispenser. Do the 2010s really require so many more policy statements than did the 1990s? I doubt it. Increasingly, they resemble products produced by a business, and so they should be treated by the general public as if that is what they are: the advertising may not always tell the full story.
As Steven Pinker argues in The Blank Slate, producing nostrums of pseudo-science to support an advocacy position is highly risky and may be counterproductive to that advocacy position in the long run. If moral arguments for equality or against corporal discipline are constructed on a foundation of faulty science, then the foundations of that morality collapse as the faults are exposed. It is far safer to ensure that moral arguments remain independent of scientific claims—to argue, for instance, that gender equality is a moral imperative irrespective of whether or not gender differences are biological, or that hitting children is wrong irrespective of whether it leads to negative outcomes for the child.
Failing to be honest about a research field’s inconsistencies and shortcomings makes the whole project look like an instrument in a fear-mongering culture war. This was the missed opportunity of the APA’s guidelines for men and boys. It would be great to have a culturally sensitive, empathic document that encouraged therapists to understand men and boys from their own perspectives. Instead, the document became lost in progressive assumptions and jargon. Discussing patriarchy and privilege with the coal miner who lost his job and can’t feed his family is unlikely to be in his (or their) best interest.
Advocacy efforts are always defined by their loudest and most extreme voices, and these can become detrimental to the cause over time as they become increasingly militant. Feminist Frequency achieved fame by pressing for better representations of female characters in video games (which I fully support; Alice: Madness Returns all the way) but then sacrificed goodwill by insinuating causal impacts on men’s attitudes and aggression toward women, another belief for which data is shaky at best. We might expect this from advocacy groups, but we expect better from organizations that claim to represent science. Unfortunately, we’re not getting it.
Social science is being poorly represented by groups like the APA and AAP. I recommend that, for the time being, parents, policy makers, and the public should, at best, treat resolution statements with a grain of salt and perhaps ignore them altogether. Changing this state of affairs would take a major cultural change at these organizations and, having been given a glimpse on the inside, it’s difficult to imagine this happening any time soon.
