Author Archives: Brent Donnellan

I helped contribute a short piece for a divisional newsletter about methodological reform issues.  I did this with three much smarter colleagues (Lucas, Fraley, and Roisman) and this is something I highly recommend.  However, I take all responsibility for the ideas in this post.

Anyways, this turned out to be an interesting chance to write about methodological reform issues and provide a “Summer Reading” list of current pieces. We tried to take a “friendly” approach by laying out the issues so individual researchers could read more and make  informed decisions on their own.  Here is a link to the ReformPrimer in draft form.  I posted the greatest hits below.

Six Principles and Practices to Consider Adopting in Your Own Work

1. Commit to Total Scientific Honesty.  See Lykken (1991) and Feynman (1985).

2. Continue reading →

I want to chime in about the exciting new section in Perspectives on Psychological Science dedicated to replication.  (Note: Sanjay and David have more insightful takes!). This is an important development and I hope other journals follow with similar policies and guidelines.  I have had many conversations about methodological issues with colleagues over the last several years and I am constantly reminded about how academic types can talk themselves into inaction at the drop of a hat. That fact that something this big is actually happening in a high profile outlet is breathtaking (but in a good way!).

Beyond the shout out to Perspectives, I want to make a modest proposal:  Donate 5 to 10% of your time to replication efforts.  This might sound like a heavy burden but I think it is a worthy goal. It is also easier to achieve with some creative multitasking.   Steer a few of those undergraduate honors projects toward a meaningful replication study or have first year graduate students pick a study and try to replicate it during their first semester on campus.  Then make sure to take an active role in the process to make these efforts worthwhile for the scientific community.  Beyond that, let yourself be curious!  If you read about an interesting study, try to replicate it. Continue reading →

I once read about an experiment in which college kids held either a cold pack or a warm pack and then reported about their levels of so-called trait loneliness. We just tried a close replication of this experiment involving the same short form loneliness scale used by the original authors. I won’t out my collaborators but I want to acknowledge their help.

The original effect size estimate was pretty substantial (d = .61, t = 2.12, df = 49) but we used 261 students so we could have more than adequate power. Our attempt yielded a much small effect size than the original (d =-.01, t = 0.111, df = 259, p = .912).  The mean of the cold group (2.10) was darn near the same as the warm group (2.11; pooled SD = .61). Continue reading →

The folks at the Language Log did a nice job of considering some recent claims about the narcissism and delusions of today’s young people. I want to piggy-back on that post with an illustration from another dataset based on work I have done with some colleagues.

We considered a JPSP paper by a group I will just refer to as Drs. Smith and colleagues. Smith et al. used data from the Monitoring the Future Study from 1976 to 2008 to evaluate possible changes in the life goals of high school seniors. They classified high school seniors from 1976 to 1978 as Baby Boomers (N = 10,167) and those from 2000 to 2008 as Millennials (N= 20,684). Those in-between were Gen Xers but I will not talk about them in the interest of simplifying the presentation.

Students were asked about 14 goals and could answer on a 1 to 4 point scale (1=Not Important to 4=Extremely Important). Smith et al. used a centering procedure to report the goals but I think the raw numbers are as enlightening.  Below are the 14 goals ranked by the average level of endorsement for the Millennials.

“Although well-meaning, many of the suggestions only make sense if you say them very fast.”   -Howard Wainer (2011, p. 7) from Uneducated Guesses

I love this phrase: Your ideas only make sense if you say them very fast. I find myself wanting to invoke this idea anytime I hear some of the counterarguments to methodological reform. For example, I think this line applies to NS’s comment about climate change skeptics.

Anyways, I am about 90% done with the articles in the November 2012 special issue of Perspectives on Psychological Science.  I enjoyed reading most of the articles and it is good resource for thinking about reform in psychological research. It should probably be required reading in graduate seminars. So far, the article that generated the strongest initial reaction was the Galak and Meyvis (2012; hereafter G & M) reply to Francis (2012).  I think they basically made his point for him.  [I should disclose that I think the basic idea pursued by G and M seems plausible and I think their reply was written in a constructive fashion. I just did not find their arguments very convincing.]

Disclaimer: Both Robert MacCallum (e.g., 2003) and George Box (e.g., 1979) have written extensively about the value of models and I will basically steal parrot their ideas in this post. Moreover, I did not sleep much last night…

Let the postmortem on the 2012 election begin! One story will likely involve the accuracy of well-conducted polls and the success of Nate Silver’s methods over “gut-based” methods favored by pundits and campaign workers.  Not surprisingly, I like much of this story as is nicely summed up by this cartoon (thanks to Skip G. for posting this one on “the” Facebook).

But what if Nate Silver was wrong? What if we woke up today and Romney won 303 electoral votes and Obama lost the election?  I think Mr. Silver would have been in a much better position than his “gut-based” critics are in today. Continue reading →

I had been wondering if political orientation or discrepancies in political orientation might be related to relationship quality. I think this is an interesting question in light of a close presidential election. Fortunately, I had access to some data on these variables from around 330 heterosexual married couples. I conducted some preliminary analyses this morning and the short story is a bunch of null findings.

Measures: Political orientation was measured on the “traditional seven-point scale” where 1=extremely liberal to 7 = extremely conservative (see Knight, 1999). Marital quality was measured using five items from the quality of marriage index (Norton, 1983).  The internal consistencies were typical of this measure (alphas ≥ .90 for wives and husbands)

Descriptive Results: Husbands were slightly more conservative than wives (Husband Mean = 4.63, Wife Mean = 4.33, Pooled SD = 1.36; d = .22). Husbands and wives did not differ in terms of marital quality (Husband Mean = 4. Continue reading →

Last winter I gave a quick brown bag where I speculated about the possibility of two distinct types of researchers. I drew from a number of sources to construct my prototypes. To be clear, I do not suspect that all researchers will fall neatly into one of these two types. I suspect these are so-called “fuzzy” types. I also know that at least one of my colleagues hates this idea. Thus, I apologize in advance.

Regardless, I think there is something to my working taxonomy and I would love to get data on these issues. Absent data, this will have to remain purely hypothetical. There is of course a degree of hyperbole mixed in here as well. Enjoy (or not)!

The Editor of Emotion does not appear to be interested in seeing the Bargh and Shalev data from Study 1a. I will probably have more to say about that decision once the dust settles and we get some clarification. Now what?

Uli Schimmack suggests that others ask for the data to have a fair hearing for our concerns (see his comment on my previous blog entry). This might be the only option left.  If the field is going to move toward more openness and transparency, I guess it will have to be a grass roots movement. So be it.

The Claim: There is something incredible about the distributions of at least one of the bathing/showering items in Study 1a of Bargh and Shalev (2012). Just seeing the distribution should lead researchers to put an asterisk on that study so it is not included in a future meta-analysis without a big caveat.

What to do:

1. Obtain the raw data. Continue reading →

Answer: You do not talk about John Bargh’s data.

I went on hiatus with back to school events and letter of recommendation writing.  However, I think this is a good story that raises lots of issues. I need to say upfront that these opinions are mine and do not necessarily reflect anyone else’s views. I might also be making a big enemy with this post, but I probably already have a few of those out there. To quote the Dark Knight: I’m not afraid, I’m angry.

Background: Bargh and Shalev (2012) published an article in Emotion where they predicted that trait loneliness would be “positively associated with the frequency, duration, and preferred water temperatures” of showers and baths (p. 156). The correlation between self-reported loneliness and self-reported “physical warmth extraction” from baths/showers was .57 in Study 1a (51 undergrads) and .37 in Study 1b (41 community members). This package received media attention and was discussed in a Psychology Today blog post with the title: “Feeling lonely? Take a warm bath. Continue reading →