Episode 2 - Examining analytic flexibility
This week discuss analytic flexibility in “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant” from Simmons, Nelson, & Simonsohn (2011) http://journals.sagepub.com/doi/abs/10.1177/0956797611417632
Highlights:
[1:00] What is a false positive anyway?
[3.30] Dead salmon fMRI study http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf
[4:30] Pregnant men as our example of false positives
[8:30] Garden of forking paths
[11:00] Gelman & Lokens (2013) http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
[18:00] precognition - we needed results that broke physics to show us the problems in psychology
[21:00] Table 1 and simulating researcher degrees of freedom
[24:30] “Our goal as scientists is not to publish as many articles as
we can, but to discover and disseminate truth.”
[27:30] you can always get significances with optional stopping
[29:30] March of the P-values https://www.youtube.com/watch?v=5OL1RqHrZQ8, ‘Improving your statistical inferences’ on Coursera with Daniel Lakens (@lakens)
[32:00] Solutions, inc. transparency and preregistration
Music credit: Kevin MacLeod - Funkeriffic
freepd.com/misc.php
