One of the many projects that foundations like to spend money on is coming up with new ideas for altering people’s behavior. This is increasingly true as the left has had a crisis of confidence, retreating from socialism as the goal to running “nudge” programs that will convince people to admire government. No one, with the possible exception of British Labour Party leader Jeremy Corbyn[1], believes in nationalizing the means of production. But far too many people think that social science can push people to do things they would otherwise not do.
But what if much of the research of behavioral science wasn’t true? What is it was little more than a giant sinkhole of flabbergab? That’s the provocative thesis of Andrew Ferguson, in this highly entertaining article in the Weekly Standard.
Ferguson begins with a discussion of a report issued this past August by the Center for Open Science’s Reproducibility Project. The project took 100 papers that had been peer reviewed and which appeared in some of the top psychological journals. They then tried to duplicate their results. Repeating experiments is a cornerstone of science; if a study can’t be reproduced, there’s something wrong with it.
The Center for Open Science found that two-thirds of the studies it looked at couldn’t be reproduced, which meant they probably weren’t true. But Ferguson argues this is the tip of the iceberg. He quotes an editorial that appeared in the British medical journal The Lancet where the magazine’s editor, Dr. Richard Horton, declared that “much of the scientific literature, perhaps half, may simply be untrue” in part because science today is “afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest with an obsession for pursuing fashionable trends of dubious importance.”
These biases, Ferguson reports, come from several sources. One is that it’s easier to get published if you have positive results, so the data is tweaked to produce the results that ensure publication. Another is that most of the test subjects for papers in the behavioral sciences are undergraduate students, who tend to be psych majors. Ferguson notes that these students “come cheap” since hundreds of them can be found on a campus “lying around with nothing else to do.” But it’s also true that universities, faced with budget cuts, restrict the test subjects who can be paid with university funds to university students.
Take one of the most famous experiments in psychology, conducted by Yale psychologist Stanley Milgram in the early 1960s, and the subject of the new film Experimenter. Milgram said that he had nearly 600 test subjects do a study where they sat in one room and turned up a dial so that a subject in another room would suffer pain. (The machine was fake; no one actually suffered.) He said that nearly 65 percent of the people he tested turned the dial, thus showing that most people obeyed authority, a finding he said accounted for the rise of Hitler.
In her book Beyond the Shock Machine, Gina Perry analyzed Milgram’s results. Milgram did use 600 people, but these came from a variety of studies and most people didn’t turn up the dial. Milgram’s findings came from a 40-man study in which all the subjects were males attending Yale who were paid $4.50. Out of the 40, 26 turned up the dial. “How we get from the 26 Yalies in a New Haven psych lab to the anti-Semitic psychosis of Nazi Germany has never been explained,” Ferguson writes. It is also possible that those who turned the dial did so, not because they were unquestioning automatons, but because that’s what they thought it would take to get paid.
A third example of bias comes from the fact that nearly all social scientists are on the left. The situation is so bad that a group of 22 social scientists, led by New York University psychologist Jonathan Haidt, has launched an effort to try to bring in more non-liberals into the social sciences.
Ferguson quotes from “Political Conservatism as Motivated Social Cognition,” a paper that has been cited over 2,000 times.
“A meta-analysis confirms that several psychological variables predict political conservatism: death anxiety; system instability; dogmatism—intolerance of ambiguity; openness to experience; uncertainty tolerance; needs for order, structure, and closure; integrative complexity; fear of threat and loss; and self-esteem.”
“Only a scientist planted deep in ideology could read such a summary and miss the self-parodic assumptions buried there,” Ferguson observes.
Does all this social science research do any good or change anyone’s mind? Ferguson cites a recent report by the Obama administration’s Social and Behavioral Sciences Team, “a cross-agency group of experts in applied behavioral science that translates findings and methods from the social and behavioral sciences into improvements in Federal policies and programs.”
The team came up with two recommendations. One was making forms that veterans used to apply for benefits simpler, so that more veterans could apply for them. Another was making forms for applying for financial aid simpler, to ensure that more applicants would fill out the forms.
After the report, President Obama issued an executive order ordering the heads of agencies to “recruit behavioral science experts to join the Federal government as necessary to achieve the goals of this directive.”
“We should have known!” Ferguson concludes. “After all the bogus claims and hyped findings and preening researchers, after the tortured data and dazed psych students, this is the final product of the mammoth efforts of behavioral science: a federal jobs program for behavioral scientists.”
What should funders learn from Ferguson’s fine article? If they get grant proposals whose ultimate purpose is to change human behavior, they should take a deep breath, walk around the block—and fund something else.
[1] As long as I’m mentioning Corbyn, let me pass on a comment by Financial Times columnist Tim Harford, who says that Corbyn is the sort of guy who would reopen the coal mines one week to keep the unions happy, then close them the following week to keep the environmentalists happy.
