Episode 8 - The preregistration revolution
It’s preregistration time! Distinguishing confirmatory (or prediction) and exploratory (or postdiction) analyses. We discuss this awesome paper from Nosek, Ebersole, DeHaven, and Mellor (2018) “The preregistration revolution”. Find the paper here http://www.pnas.org/content/early/2018/03/08/1708274114
Highlights
[1:00] What is a preregistration anyhow?
[2:00] Prediction vs postdiction
[3:00] Why is preregistration important? Combatting bias; distinguishing confirmatory and exploratory analyses.
[4:45] In a perfect world, do we need preregistration?
[10:00] What else does preregistration guard against, or not?
[12:00] Getting critical; does prereg always work? Probably not, we need quality control. So, is prereg better than nothing? maybe not…
[14:30] Where are your preregistrations on the preregistration spectrum?
[16:00] Registered reports - yes, we mention them several times. Do we define or describe them, not at all…
[17:00] Is preregistration the ‘open science thing’ people are most likely to know about and use in their research, compared to other open science initiatives?
[24:00] Battle royale: preregistered and messy Vs clean and novel - which is more publishable?
[27:30] Preregistration and statistics - it’s called hypothesis testing for a reason.
[31:00] “All of a sudden we did 20 tests…”
[33:00] Challenges to preregistering your analyses
[39:00] We love preregistration, but that does not mean that it’s a simple and easy solution
[40:00] Preregistrations for existing data (SIPS hack-a-thon on secondary data analyses https://osf.io/djgvw/)
[40:00] Preregistrations for qualitative data (template https://osf.io/j7ghv/)
[43:00] Listener challenge - tell us the terrible reasons you’ve heard against preregistration
[48:00] What’s next for reproducibiliTea?
Music credit: Kevin MacLeod - Funkeriffic
freepd.com/misc.php
