i have found the solution and it is us – Simine Vazire (sometimes i'm wrong)

 [DISCLAIMER: The opinions expressed in my posts are personal opinions, and they do not reflect the editorial policy of Social Psychological and Personality Science or its sponsoring associations, which are responsible for setting editorial policy for the journal.]

Happydog bear, having recently joined SIPS

i have found scientific utopia.*

sometimes, when i lay awake at night, it's hard for me to believe that science will ever look the way i want it to look,** with everyone being skeptical of preliminary evidence, conclusions being circumscribed, studies being pre-registered, data and materials being open, and civil post-publication criticism being a normal part of life.
then i realized that utopia already exists.  it's how we treat replication studies.
i've never tried to do a replication study,*** but some of my best friends (and two of my grad students) are replicators.  so i know a little bit about the process of trying to get a replication study published.  short version: it's super hard.
we (almost always) hold replication studies to an extremely high standard.  that's why i'm surprised whenever i hear people say that researchers do replications in order to get an 'easy' publication.  replications are not for the faint of heart.  if you want to have a chance of getting a failed replication**** published in a good journal, here's what you often have to do: Continue reading

Everything is fucked: The syllabus – Sanjay Srivastava (The Hardest Science)

PSY 607: Everything is Fucked
Prof. Sanjay Srivastava
Class meetings: Mondays 9:00 – 10:50 in 257 Straub
Office hours: Held on Twitter at your convenience (@hardsci)

In a much-discussed article at Slate, social psychologist Michael Inzlicht told a reporter, “Meta-analyses are fucked” (Engber, 2016). What does it mean, in science, for something to be fucked? Fucked needs to mean more than that something is complicated or must be undertaken with thought and care, as that would be trivially true of everything in science. In this class we will go a step further and say that something is fucked if it presents hard conceptual challenges to which implementable, real-world solutions for working scientists are either not available or routinely ignored in practice.

The format of this seminar is as follows: Each week we will read and discuss 1-2 papers that raise the question of whether something is fucked. Our focus will be on things that may be fucked in research methods, scientific practice, and philosophy of science. The potential fuckedness of specific theories, research topics, etc. will not be the focus of this class per se, but rather will be used to illustrate these important topics. To that end, each week a different student will be assigned to find a paper that illustrates the fuckedness (or lack thereof) of that week’s topic, and give a 15-minute presentation about whether it is indeed fucked.

Grading:

20% Attendance and participation
30% In-class presentation
50% Final exam

Continue reading

The Personality Parsimony Myth

I woke one recent morning to this text from my girlfriend: “Today’s episode of Invisibilia is called ‘The Personality Myth’!” She is, like many extraverts, easily enthused,[1] so there was no way to tell if the exclamation point indicated glee or dismay. Both emotions would’ve been reasonable. Our field — personality psychology — doesn’t get much coverage from science journalists. Nor should it, I suppose. We don’t search for new planets or cure epidemics or mash together previously unobserved physical matter. Still, use of the word “myth” was unsettling… “a widely held but false belief or idea.” Continue reading →

The Mating Strategies of Extraverts – Scott McGreal (Unique—Like Everybody Else)

Life History Theory views long-term and short-term mating strategies as polar opposites. However, some people, particularly extraverts, may pursue both, others neither.

don’t you know who i am? – Simine Vazire (sometimes i'm wrong)

 [DISCLAIMER: The opinions expressed in my posts are personal opinions, and they do not reflect the editorial policy of Social Psychological and Personality Science or its sponsoring associations, which are responsible for setting editorial policy for the journal.]

Elephantseal05elephant seal, throwing his weight around

when i started my first job as associate editor, i was worried that i would get a lot of complaints from disgruntled authors.  i wasn't afraid of the polite appeals based on substantive issues, i was worried about the complaints that appeal to the authors' status, the "don't you know who i am?" appeal.

i never did get that kind of response, at least not from authors. but i saw something worse - a pretty common attitude that we should be judging papers based, in part, on who wrote them.  socially sanctioned status bias. not so much at the journals i worked with, but in the world of journals more broadly. like the Nature editorial, on whether there should be author anonymity in peer review, that argued that "identifying authors stimulates referees to ask appropriate questions (for example, differentiating between a muddy technical explanation and poor experimental technique)." the argument seems to be that some people should be given a chance to clear up their muddy explanations and others should not. or the editor who wrote in The Chronicle of Higher Education just a few days ago that "Editors rarely send work out to trusted reviewers if it comes from unproven authors using jazz-hands titles."  leaving aside the contentious issue of jazz-hand titles, when did we accept that it was ok to treat papers from 'unproven authors' differently? Continue reading

Don’t change your family-friendly tenure extension policy just yet – Sanjay Srivastava (The Hardest Science)

pixelated something If you are an academic and on social media, then over the last weekend your feed was probably full of mentions of an article by economist Justin Wolfers in the New York Times titled “A Family-Friendly Policy That’s Friendliest to Male Professors.” It describes a study by three economists of the effects of parental tenure extension policies, which give an extra year on the tenure clock when people become new parents. The conclusion is that tenure extension policies do make it easier for men to get tenure, but they unexpectedly make it harder for women. The finding has a counterintuitive flavor – a policy couched in gender-neutral terms and designed to help families actually widens a gender gap. Except there are a bunch of odd things that start to stick out when you look more closely at the details, and especially at the original study. Let’s start with the numbers in the NYT writeup:

The policies led to a 19 percentage-point rise in the probability that a male economist would earn tenure at his first job. In contrast, women’s chances of gaining tenure fell by 22 percentage points. Before the arrival of tenure extension, a little less than 30 percent of both women and men at these institutions gained tenure at their first jobs.

Two things caught my attention when I read this. Continue reading

your inner third grader – Simine Vazire (sometimes i'm wrong)

 [DISCLAIMER: The opinions expressed in my posts are personal opinions, and they do not reflect the editorial policy of Social Psychological and Personality Science or its sponsoring associations, which are responsible for setting editorial policy for the journal.]

Embarrassedbear

it felt like a confessional. 'sometimes, we say we predicted things that we didn't actually predict.'  i paused, embarrassed.  'i know.' 'i'm sorry,' she said, 'but that sounds like something even a third grader would know is wrong.' 'i know.' i tried not to make excuses, but to explain how this happened.  how an entire field convinced itself that HARKing (Hypothesizing After the Results are Known, Kerr, 1998) is ok. Continue reading