Happiness and Academic Malpractice

In two popular books about happiness, Professor Paul Dolan of the London School of Economics has established himself as a public scholar. Dolan writes very clearly—an unusual attribute for an academic—and brings a fresh approach to the study of happiness. Quillette founder Claire Lehmann captures it well in a review of Dolan’s first book, Happiness by Design:

When we think about our “happiness” we may think about the goals we have achieved, how much money we have in the bank, or how prestigious our job is. We may not think about our commute to work, our dreary co-workers or the fact that days at the office seem to drag along, uninspiringly. In doing so, Dolan argues, we privilege our evaluative self over the experiential self.

Dolan’s exploration of the experiential self isn’t intended just to be a theory: he presents statistical data from numerous sources in support of his hypothesis. Unfortunately, Dolan’s data provide very little support for his theory, and are marred by serious errors of both analysis and interpretation.

Dolan first caught my attention on May 25, 2019, when the Guardian published a quote from him that struck me as unusually strident for a social scientist: “Married people are happier than other population sub-groups but only when their spouse is in the room when they’re asked how happy they are. When the spouse is not present: fucking miserable.” This ran counter to all the data I’ve seen in my decades of being a family scholar, data that consistently show that married people are happier than their unmarried contemporaries. At the time, Dolan had been publicizing his latest book, Happy Ever After.

Writing for the Family Studies Blog on May 28, I provided evidence that Dolan’s data didn’t really support his contention. His claims about the fucking misery of marriage rely upon data from the American Time Use Survey. These data are shown Figure 10 of Happy Ever After. I emailed Dolan to inquire about this figure on May 25, 2019, but as of June 8 he has not responded.

Below is a mock-up of the figure, with elements missing to avoid copyright law hot water:

The figure is measuring unhappiness by marital status on a scale from 0 (not unhappy) to 6 (very unhappy). The y-axis scale is faithful to Happy Ever After. Contrary to what Dolan implies in the Guardian, the figure covers both wives and husbands. Each of the six bars corresponds to a different marital status, with married respondents split up into “spouse present” and “spouse absent” categories:

• Married, spouse present
• Married, spouse absent
• Widowed
• Divorced
• Separated
• Never married

Looking at the y-axis, it’s pretty clear that few survey respondents are unhappy. Indeed, the .2 difference between bars represents the entirety of the difference between “happy” and “fucking miserable” as represented by Dolan in the Guardian article. This is a pretty trivial difference by any stretch of the imagination.

Is the difference statistically significant? Dolan doesn’t say. Are these results adjusted for socio-demographic differences between respondents? Dolan doesn’t say that either. Are both heterosexual and same-sex marriages depicted in Figure 10? More silence from Dolan.

Indeed, Dolan’s book contains very few methodological details. He is candid about not doing the data analysis himself; instead, he’s drawing on the dissertation of his graduate student and frequent collaborator Laura Kudrna. (This dissertation is freely available online for anyone curious.)

Dolan’s book, in short, doesn’t appear to support what he said in the Guardian interview. This is the main story I told in my Family Studies article. It turned out I’d only fired the first salvo. United States government economist Gray Kimbrough, well acquainted with the American Time Use Survey, soon identified an even more damning problem with Dolan’s work.

Happiness by Design - Paul Dolan

Dolan’s proposal is that shifting our attention away from constructed narratives to actual experiences is likely to make us a lot happier.

QuilletteClaire Lehmann

Dolan had erroneously taken “Married, spouse absent” to mean that a respondent’s spouse just wasn’t in the room. In fact, Kimbrough pointed out, it meant that the spouse wasn’t even in the household; it referred to spouses living separately. Dolan apparently hadn’t taken the time to learn what his data were actually measuring. One Twitter wag suggested that “spouse present” was the new “death recorded” (a reference to Naomi Wolf’s embarrassing flub in interpreting nineteenth century British law).

Kimbrough systematically dismantled other Dolan claims in a Twitter thread. Dolan’s suggestion that marriage isn’t related to women’s physical health is refuted by the earlier research he cites in his book. The American Time Use Survey data he invokes offers no evidence in support of the contention that the “healthiest and happiest population subgroup are women who never married or had children.” Moreover, my own analysis of data from the General Social Survey supports Kimbrough’s findings—and repudiates Dolan’s. The figure below shows that married women are much more likely to report being “very happy” than are their previously married or never-married contemporaries. The only part of the story that Dolan appears to get right is the claim that people with kids are a bit less happy.

Elsewhere Dolan offers claims that can be refuted just as easily. For instance, in a portion of his book excerpted by the Guardian in January of this year, he claims that people who make over $100,000 a year are no happier than those making under $25,000.

General Social Survey data give lie to Dolan’s claims. The data below show that people with incomes above $100,000 are decidedly happier than are those who make less than $25,000 a year. What’s more, income and marital status continue to be correlated with happiness even after controlling for the usual social and demographic suspects: education, race, age, and so on.

Academics make mistakes, even dumb ones. Dolan could have easily come clean about everything, but he didn’t. He offered this half-denial to journalist Charles Lehmann:

My update from this am: pic.twitter.com/5U1X0AgjDN

— Charles Fain Lehman (@CharlesFLehman) May 31, 2019

The Guardian struck the “fucking miserable” claim from the May 25 article, but included no retraction notice.

Perhaps more noteworthy, Dolan stuck to his guns about marriage being better for men, despite having the rug pulled out from under him by both me and Gray Kimbrough. Indeed, he doubled down in an article he wrote for the Guardian on June 4, then tried to misdirect his critics attention with a speculative digression about stigma against single people. In a BBC interview on June 7, Dolan was asked point blank (at 6:58) if he wanted to retract his claims about marriage being “fucking miserable.” He neither affirms nor retracts; he dissembles. At 8:08, the interviewer said, “Well, the interview went on for a long time, but I’ll spare you, because I never did manage to get Paul Dolan to produce any evidence that married women were flipping [sic] miserable.” This had been preceded by an interview with Grey Kimbrough in the same segment.

Dolan’s “findings” about marriage were spread far and wide in the days after the May 25 Guardian article. The Guardian itself repeated the fiction two days later (and again, two days after that), but couldn’t be bothered to correct it. Meanwhile, the story had already popped up in clickbait at the New York Post, the UK’s Telegraph, and myriad other sites. Far fewer have set the story straight (two exceptions are the aforementioned BBC segment and a detailed explainer in Vox).

As readers of Quillette well know, some tempest in a teacup is always spilling out of academia into the broader media. Why should we care about Paul Dolan’s academic misfeasance? Canadian economist Marina Adshade got it right with this assessment on Twitter:

Paul Dolan’s specious science undermines public confidence in higher education, already at a low ebb in the United States and elsewhere. Conservative politicians have stepped up their rhetorical and legislative onslaught in recent years. “We’ll take $200,000 of your money; in exchange we’ll train your children to hate our country,” said the president’s son, Donald Trump Jr., in 2017. Meanwhile, the “Sokal Squared” hoax has revealed that a small sector of higher education is engaged in an intellectually dubious enterprise.

A lot of science is flawed in one way or another, but few of the errors are as careless as Dolan’s. He compounded his sins by repeatedly failing to take responsibility for them. If nothing else, Paul Dolan is a cautionary tale for young academics in how they should engage with the press. The rest of us have a moral responsibility to speak up the next time a huckster makes the news.