21 Jul 2016
We recently reported an attempt to replicate (Hardwicke et al., 2016) a previous demonstration of reconsolidation-mediated memory updating in human participants (Walker & Stickgold, 2003). The authors of the original study have written a commentary on the paper (Walker & Stickgold, 2016) and we have responded (Hardwicke & Shanks, 2016). The discussion focuses on whether our non-replications established ‘boundary conditions’ on the memory reconsolidation theory. Although there was little space to fully develop the arguments, the discussion touches on some interesting philosophical issues about how replications relate to theory. I’d be very interested to hear people’s thoughts!
Hardwicke, T. E., Mahdi, T., & Shanks, D. R. (2016). Post-retrieval new learning does not reliably induce human memory updating via reconsolidation. Proceedings of the National Academy of Sciences of the United States of America, 113, 5206-5211. [doi] [osf] [pdf]
Walker, M. P., & Stickgold, R. (2016). Understanding the boundary conditions of memory reconsolidation. Proceedings of the National Academy of Sciences of the United States of America, 113, 1-2. [doi] [pdf]
Hardwicke, T. E., & Shanks, D. R. (2016). Reply to Walker and Stickgold: Proposed boundary conditions on memory reconsolidation will require empirical verification. Proceedings of the National Academy of Sciences of the United States of America, 113, 1-2. [doi] [pdf]
12 May 2016
Our meta-science project evaluating the effectiveness of the Open Badges scheme was published today in PLoS Biology (Kidwell et al., 2016). The idea behind badges is simple: when a paper is accepted for publication, the editor contacts the authors and asks if they would like to make their data and materials publicly available on a 3rd party repository (like the OSF). If the authors decide to go for it (the scheme is voluntary), then a colourful badge will appear at the top of their published manuscript signaling the open practice, along with a download link for the relevant data/materials.
Simple! And apparently highly effective. Here are two of the key figures, showing a dramatic increase in the quantity and quality of data-sharing at the journal Psychological Science after Open Badges were introduced in Jan 2014 (comparison journals are also shown).
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S., & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low cost, effective method for increasing transparency. PLoS Biology, 14, 1-15. [doi] [osf] [pdf]
25 Apr 2016
In a new paper published in PNAS (Hardwicke et al., 2016) we report a number of attempts to replicate a previous demonstration of reconsolidation-mediated memory updating in human participants. Here is the key figure, showing the we did not observe reconsolidation effects in four direct replication attempts.
15 Apr 2016
In a new commentary published in JEP:G (Vadillo et al., 2016) we examine why ‘money priming’ effects have proved elusive in a number of replication attempts (Rohrer et al., 2015; Klein et al., 2014) when there appears to be a large body of literature reporting evidence in favour of the money priming hypothesis (Vohs, 2015).
We found that a range of meta-analytic tools suggest irregularities in the money priming literature that may indicate considerable selective reporting and/or p-hacking. This could explain why money priming effects are consistently absent in pre-registered and transparently reported direct replications that minimise potential biases.
06 Jan 2016
A project I am involved with “The Pre-Registration Challenge” has officially launched today - its a great opportunity for you to try out a new methodological approach in the spirit of open science.
Full details about entering the challenge can be found here:
If you decide to enter, you will be walked through a step-by-step pre-registration (specifying methods and analysis prior to data collection) on The Open Science Framework. As a small incentive, you could also win $1000…
If you’re in the London area and umming and ahhing about the whole thing, then as a further incentive I will happily take you out for a coffee and a pre-registration chat. I’ll pay for the coffee even if you don’t decide to try out pre-registration, so you have nothing to lose!