The Pre-Publication Transparency Checklist: A Small Step Toward Increasing the Believability of Psychological Science
We now know that some of the well-accepted practices of psychological science do not produce reliable knowledge. For example, widely accepted but questionable research practices contribute to the fact that many of our research findings are unbelievable (that is, that one is ill-advised to revise one’s beliefs based on the reported findings). Post-hoc analyses of seemingly convincing studies have shown that some findings are too good to be true. And, a string of seminal studies have failed to replicate. These factors have come together to create a believability crisis in psychological science.
Many solutions have been proffered to address the believability crisis. These solutions have come in four general forms. First, many individuals and organizations have listed recommendations about how to make things better. Second, other organizations have set up infrastructures so that individual researchers can pre-register their studies, to document hypotheses, methods, analyses, research materials, and data so that others can reproduce published research results (Open Science Framework). Third, specific journals, such as Psychological Science, have set up pre-review confessionals of sorts to indicate the conditions under which the data were collected and analyzed. Fourth, others have created vehicles so that researchers can confess to their methodological sins after their work has been published (psychdisclosure.org). In fact, psychology should be lauded for the reform efforts it has put forward to address the believability crisis, as it is only one of many scientific fields in which the crisis is currently raging, and it is arguably doing more than many other fields. Continue reading
My brain may still be in a fog from all the food I ate yesterday, but that isn't going to stop me from being thankful. I'm thankful for a great many things in my life: My family, my health, and my job are three things that first come to mind. I am especially thankful for my daughter Zoe, who just turned 8 months old last week, and is, pretty much, the best baby in all the universe (admittedly, I haven't been EVERYWHERE in the universe, but I think it's at least a fair hypothesis with some empirical support). When I think about Zoe growing up, I wonder about the kind of person she is going to be and the things she is going to be interested in doing for her life. Along with these thoughts, I worry about whether Zoe's interests will conflict with what the world around her says about what she can or cannot do. If she wants to go into science, for instance, will there be people or institutions telling her that science simply isn't a thing that "people like her" are interested in? Thinking about this must be raising my blood pressure.
You're an intelligent bunch, PYM readers, so I don't need to review all the details, but when women pursue science careers they face barriers that men do not. These barriers include norms and expectations that socialize men and women to think that a science career is only compatible with the male gender, unwanted sexual advances from superiors (typically men) who make the science environment a hostile workplace (here
), and direct and indirect discriminatory practices that make it more difficult for women to succeed in a science career (here
for an example, and here
behind a paywall).
And yet, despite these significant obstacles, women still pursue science careers and excel! Today, I would like to give thanks to my female role models in psychological science. These are female scientists who have shaped my research career and through their own path-breaking work, have made science more accessible to women everywhere!Read More->
The theory of Multiple Intelligences suggests that everyone can be "intelligent" in some way even if they do not have a high IQ. As appealing as this idea is to egalitarian sentiments, the theory has never been validated and is not supported by any empirical research.
Over the past two years, my scientific computing toolbox been steadily homogenizing. Around 2010 or 2011, my toolbox looked something like this:
- Ruby for text processing and miscellaneous scripting;
- Python/Numpy (mostly) and MATLAB (occasionally) for numerical computing;
- MATLAB for neuroimaging data analysis;
- R for statistical analysis;
- R for plotting and visualization;
- Occasional excursions into other languages/environments for other stuff.
In 2013, my toolbox looks like this:
- Python for text processing and miscellaneous scripting;
- Python (NumPy/SciPy) for numerical computing;
- Python (Neurosynth, NiPy etc. Continue reading
A while back a colleague forwarded me this quote from Stanley Schachter (yes that Stanley Schachter):
“This is a difference which is significant at considerably better than the p < .0001 level of confidence. If, in reeling off these zeroes, we manage to create the impression of stringing pearls on a necklace, we rather hope the reader will be patient and forbearing, for it has been the very number of zeros after this decimal point that has compelled us to treat these data with complete seriousness.”
The quote comes from a chapter on birth order in Schachter’s 1959 book The Psychology of Affiliation. The analysis was a chi-square test on 76 subjects. The subjects were selected from 3 different experiments for being “truly anxious” and combined for this analysis. True anxiety was determined if the subject scored at one or the other extreme endpoint of an anxiety scale (both complete denial and complete admission were taken to mean that the subject is “truly anxious”), and/or if the subject discontinued participation because the experiment made them feel too anxious.
In the latest issue of the ARP newsletter, Kelci Harris writes about diversity in ARP. You should read the whole thing. Here’s an excerpt:
Personality psychology should be intrinsically interesting to everyone, because, well, everyone has a personality. It’s accessible and that makes our research so fun and an easy thing to talk about with non-psychologists, that is, once we’ve explained to them what we actually do. However, despite what could be a universal appeal, our field is very homogenous. And that’s too bad, because diversity makes for better science. Good research comes from observations. You notice something about the world, and you wonder why that is. It’s probably reasonable to guess that most members of our field have experienced the world in a similar way due to their similar demographic backgrounds. This similarity in experience presents a problem for research because it makes us miss things. How can assumptions be challenged when no one realizes they are being made? What kind of questions will people from different backgrounds have that current researchers could never think of because they haven’t experienced the world in that way?
In response, Laura Naumann posted a letter to the ARP Facebook wall. Continue reading