Author Archives: Brent Roberts

I’m having a hard time reading scientific journal articles lately.  No, not because I’m getting old, or because my sight is failing, though both are true.  No, I’m having trouble reading journals like JPSP and Psychological Science because I don’t believe, can’t believe the research results that I find there.

Mind you nothing has changed in the journals. You find tightly tuned articles that portray a series of statistically significant findings testing subtle ideas using sample sizes that are barely capable if detecting whether men weigh more than women (Simons, Nelson, & Simonsohn, 2013). Or, in our new and improved publication motif, you find single, underpowered studies, with huge effects that are presented without replication (e.g., short reports).  What’s more, if you bother to delve into our history and examine any given “phenomena” that we are famous for in social and personality psychology, you will find a history littered with similar stories; publication after publication with troublingly small sample sizes and commensurate, unbelievably large effect sizes.  As we now know, in order to have a statistically significant finding when you employ the typical sample sizes found in our research (n = 50), the effect size must not only be large, but also overestimated.  Couple that with the fact that the average power to detect even the unbelievably large effect sizes that we do report is 50% and you arrive at the inevitable conclusion that our current and past research simply does not hold up to scrutiny. Thus, much of the history of our field is unbelievable.  Or, to be bit less hyperbolic, some unknown proportion of our findings can’t be trusted.  That is to say, we have no history, or at least no history we can trust. Continue reading →

Just recently, the Chronicle of Higher Education ran a piece on what it takes to be successful in an academic career.  It was a pleasant essay, which emphasized some of the usual suspects like industriousness and the like, but it felt a little off to me.  The thing that seemed lacking was the disparate, often conflicting nature of the profile of qualities that I believe (read: I don’t have much data to back these opinions up) are found in successful academics.  To put it in psychometric jargon, the attributes necessary for success are orthogonal—they don’t all come neatly bundled in each person.  This makes finding the combination in any given individual, which is often necessary for success, a rare occurrence.

I thought it might be constructive to start a provisional list of qualities that appear to lead to success in academic careers from my idiosyncratic vantage point.  Keep in mind that this is a theory in search of data.  The list can be used in several ways.  First, it can be used in a study to see if I’m right.  Second, it can be used as an aid in selection—self or otherwise.  Or, if you are already in the career track the list can be used for development.  I do not possess all of these qualities—very few people ever do—but I have worked hard to develop them. Alternatively, if you don’t want to change, a brilliant option is to team up with someone who complements your strengths and weaknesses.  You can be the idea person and your colleague the quant jock or vis versa. Continue reading →

The good:

Our European Journal of Personality paper proposing ways to improve the replicability of research in psychology is in press.  You can find a copy here.  The EJP editor is soliciting comments and the entire package should be published soon.

The really good:

A slew of papers on replicability in psychological science is now available in the November issue Perspectives on Psychological Science.  The entire issue is a must read.

Greg Francis has another paper in Psychonomic Bulletin and Review that provides a most brilliant analysis of why even simple, direct replication is not an answer to our problems.

And, as many of you have seen, the editors at Psychological Science circulated a letter outlining some reasonable, if modest proposals for changes to the journal that are described succinctly by Sanjay Srivastava on his blog, The Hardest Science.

For the first time in a long while, things might actually be moving in the right direction. I’m sure Nate Silver could have predicted that. Continue reading →

Some of you know that some of us (namely Brent), are more than skeptical about “gene for” studies linking candidate genes to personality and/or any other phenotype.  What you may not know is that Brent is now enthusiastic (for the moment) about gene expression research–the product of genes, not the genes themselves (i.e., RNA).

So, this week Brent re-presented a presentation he gave last week to a group of social scientists and biologists telling the tale of how he went from gene-by-environment enthusiast, to gene-by-environment skeptic, to gene expression enthusiast.  If you are interested in the presentation you can find it here.

If you are interested in reading some of the background literature Gibson’s recent paper gives you the scoop on common versus rare versus infinitesimal models of gene functioning and Munafo & Flint give you the sad news that none of the existing gene association studies in personality psychology should be paid much attention.  Ouch.

On the flip side, Steve Cole gives you the overview of Stress Genomics, which is the new kid on the block, and Miller et al show you how gene expression works.  Let’s hope their fate is unlike the previous 20 years of “gene for” studies which can now be used to line that old bird cage.

It has been a bad week for cycling, with Lance Armstrong’s tacit acknowledgement that he cheated his way to 7 Tour de France titles.  As an avid recreational cyclist who did his own turn at Lance Armstrong hero-worshiping, I am admittedly saddened by the final turn in the long sordid tale of cheating in professional cycling.

Similarly, it has been a bad summer for social psychology, with 2 more cheating scandals of our own that followed closely on the heels of the Stapel affair.  In pondering the two fields it suddenly struck me that we are living weird parallel lives.  Like cycling, many of the best and brightest researchers have been cheating their way to fame and fortune in psychology.

EPO vs SEA

Social Psychology is actually worse than cycling in two ways.  There was no fraud in cycling.  Lance Armstrong never faked a ride up the L’alpe d’huez or created a hologram version of himself that he magically rode to impossible speeds in cycling time trials.  No, Lance, like many cyclists in his cohort, apparently cheated by doing things to enhance his performance.  But he still had to put out physical effort that few of us could fathom in order to make it to the top of the hill faster than his competitors. Who, like Lance, were most likely cheating too.  The drugs of choice in cycling appeared to be EPO, or blood doping, or some variant of testosterone, all of which made riders stronger, faster, and better able to recuperate from their labors. Continue reading →