When Are Direct Replications Necessary? – Ryne Sherman (Sherman's Head)

We are told that replication is the heart of all sciences. As such, psychology has recently seen numerous calls for direct replication. Sanjay Srivastava says that replication provides an opportunity to falsify an idea (an important concept in science, but rarely done in psychology). Brian Nosek and Jeffrey Spies suggest that replication would help identify “manufactured effects” rapidly. And Brent Roberts proposed a three step process, the last of which is a direct replication of any unique study reported in the package of studies. Not everyone thinks that direct replications are useful though. Andrew Wilson has argued that replication will not save psychology and better theories are needed. Jason Mitchell has gone so far as to say that failed replications offer nothing to science as they are largely the result of practical mistakes on the part of the experimenters. So are direct replications necessary? My answer is a definitive: sometimes. Let’s start by considering what I gather to be some of the main arguments for direct replications.

Sample Sizes in Personality and Social Psychology – Brent Roberts (pigee)

R. Chris Fraley Imagine that you’ve a young graduate student who has just completed a research project. You think the results are exciting and that they have the potential to advance the field in a number of ways. You would like to submit your research to a journal that has a reputation for publishing the highest caliber research in your field. How would you know which journals are regarded for publishing high-quality research? Traditionally, scholars and promotion committees have answered this question by referencing the citation Impact Factor (IF) of journals. But as critics of the IF have noted, citation rates per se may not reflect anything informative about the quality of empirical research. A paper can receive a large number of citations in the short run because it reports surprising, debatable, or counter-intuitive findings regardless of whether the research was conducted in a rigorous manner. In other words, the citation rate of a journal may not be particularly informative concerning the quality of the research it reports. What would be useful is a way of indexing journal quality that is based upon the strength of the research designs used in published articles rather than the citation rate of those articles alone. Continue reading

open letter to editors – Simine Vazire (sometimes i'm wrong)

 

  Pen

dear editors, i love journals.  i love editors.  i love editors of journals.  that's why i want to help.  we need more quality control in our journals, and you are the ones who can do it. in july 2012, in a comment to a blog post, chris fraley wrote 'What we might need, in other words, is a formal “consumer reports” for our leading journals.' i was so excited by this idea that i wrote to him and told him it was the best actionable idea that has come out of the replicability discussion.* fast forward 27 months, and our paper, 'N-Pact Factor: Evaluating the quality of empirical journals with respect to sample size and statistical power' is out. you can read the actual paper here, and you can read chris's blog post about it here. Continue reading

What do Heroes and Psychopaths Have in Common? – Scott McGreal (Unique—Like Everybody Else)

Recent research looks at whether heroes and psychopaths are "twigs from the same branch." People who have performed heroic actions to help others often have a history of antisocial behavior as well. There may be loose connections between heroism and having "psychopathic" qualities, but the reasons for this remain unclear. read more

Guest Post by John Doris – Simine Vazire (sometimes i'm wrong)

with permission, an excerpt from john doris's forthcoming book. i wanted to post this because i think it is an excellent summary of the current crisis in social/personality psychology from a well-informed 'outsider'.  this will be review for many psychologists, but should be useful for new researchers, and for those outside of psychology who want some background/analysis from someone who is not in the trenches but has read and thought a lot about this literature and these issues.  

CouchPillowCropped

art by melissa dominiak

Doris, J. M. 2015. Talking to Our Selves: Reflection, Ignorance, and Agency. Oxford:Oxford University Press.   […]  At this writing, social psychology is being shaken by charges that many published findings, including numerous iconic findings, do not replicate when tested by independent investigators. Continue reading

How to Flunk Uber: A Guest Post by Bob Hogan – David Funder (funderstorms)

How to Flunk Uber by Robert Hogan Hogan Assessment Systems Delia Ephron, a best-selling American author, screenwriter, and playwright, published an essay in the New York Times on August 31st, 2014 entitled “Ouch, My Personality, Reviewed”  that is a superb example of what Freud called “the psychopathology of everyday life.”  She starts the essay by noting that she recently used Uber, the car service for metrosexuals, and the driver told her that if she received one more bad review, “…no driver will pick you up.”  She reports that this feedback triggered some “obsessive” soul searching:  she wondered how she could have created such a bad score as an Uber passenger when she had only used the service 6 times.  She then reviewed her trips, noting that, although she had often behaved badly (“I do get short tempered when I am anxious”), in each case extenuating circumstances caused her behavior.  She even got a bad review after a trip during which she said very little:  “Perhaps I simply am not a nice person and an Uber driver sensed it.” The essay is interesting because it is prototypical of people who can’t learn from experience.  For example, when Ms. Ephron reviewed the situations in which she mistreated Uber drivers, she spun each incident to show that her behavior should be understood in terms of the circumstances—the driver’s poor performance—and not in terms of her personality.  Perhaps situational explanations are the last refuge of both neurotics and social psychologists? In addition, although the situations changed, she behaved the same way in each of them:  she complained, she nagged and micro-managed the drivers, she lost her temper, and she broadcast her unhappiness to the world. Continue reading

DMT: Gateway to Reality, Fantasy or What? – Scott McGreal (Unique—Like Everybody Else)

The bizarre phenomenon of encounters with non-human entities under the influence of DMT has inspired both mystical and scientific speculations. Greater understanding of the psychological and personality characteristics of DMT users might help shed light on this curious phenomenon. read more