Shows
ReproducibiliTea PodcastS4E8 ReproducibiliTea Reading ListsIn this episode, Will is joined by Lianne Wolsink, PhD candidate at Ruhr University Bochum and current steering committee member of ReproducibiliTea. Will and Lianne discuss the ReproducibiliTea reading lists, created to help journal clubs do deep dives on metascience topics. Lianne created reading lists on replication and science communication; Will created an introductory reading list on Open Science, preregistration, and theory in psychology.
Reading lists: https://rpt-rl.netlify.app
Books mentioned:
Science Fictions by Stuart J Ritchie
Against Method by Paul Feyerabend
Nobody's Fool by Daniel Simons and Christopher Chabris2025-06-1053 minReproducibiliTea PodcastS4E7 Critical theory vs positivismShow notes:
OSF Preprints | Let's talk about it: positivism and critical theory in dialogue https://osf.io/preprints/osf/t2ywh_v1
Caliban & the Witch by Silvia Federici
Pollution is Colonialism by Max Liboiron
Against Method by Paul Feyerabend
Lecture on Against Method by Tomas Petricek, University of Cambridge : https://youtu.be/jzD1O_gastA
S2E10: Limits of ReproducibiliTea for discussion and resources of a feminist lens on open science https://soundcloud.com/reproducibilitea/s1e10-limits
The battle for your brain by Nita Farahany
Quokka app: https://palm-lab.github.io/QualCA
Quokka video guide: https://www.youtube.com...2025-05-0652 min
The Science Basement Podcast5.3 ReproducibiliTea: Grassroots Action for Open Science w/ Vootele, Anastasiia, & SujaiThe last decade has been called the decade of the reproducibility crisis—but this isn’t just a temporary flare-up. It’s a chronic, structural issue that cuts across disciplines and impacts the very foundation of how we produce and trust scientific knowledge. In this episode, we speak with members of the ReproducibiliTea Journal Club, a grassroots movement that empowers early-career researchers to tackle problems in reproducibility head-on by building local communities focused on open, transparent science.
Together, we explore how flawed incentives, poor data practices, and a culture of publishing over precision have led to a growing erosion of trust...2025-05-0248 min
ReproducibiliTea PodcastS4E6 Creating Open-Access Scientific Software with Josh de Leeuw (jsPsych)In this episode, Sarah and Will chat to Josh de Leeuw from Vassar College and the creator of jsPsych. We chat about the history of jsPsych, the unseen process behind creating open-access scientific software, and the current challenges facing software developers in the open scholarship movement.
jsPsych is a javaScript framework for creating online experiments, and is always looking for people to contribute to the codebase: https://jspsych.org.
Follow Josh de Leeuw on Bluesky: https://bsky.app/profile/joshdeleeuw.bsky.social2025-03-2559 minReproducibiliTea PodcastS4E5 Indigenous Research Methods: Valuing Knowledge, Community, and ImpactWhy is Indigenous health research so crucial, especially for adolescents? What are the
systemic challenges researchers face, and how can we push for openness, transparency,
and equity in health research? In this conversation, hosts Queen Saikia and Sarah Sauvé
speak with leading experts from New Zealand - Prof. Sue Crengle (University of Otago),
Prof. Terryann Clarke (University of Auckland), and Dr. Andrew Sise (University of
Otago) - to explore the current landscape of Indigenous health research and the
methodologies used in these investigations. They also discuss key issues around
reproducibility, openness, and transparency, the discrepancies in research systems, and
how w...2025-02-1253 minReproducibiliTea PodcastS4E4 Science Communication (with Jamie Moffa, In Plain English Podcast)In this episode, Will is joined by Jamie Moffa, a doctoral student in systems neuroscience at Washington University in St. Louis. Jamie has been thinking and working in the science communication space, especially via the In Plain English podcast, which is aimed at bringing scientific knowledge and understanding to the general public.
Show Notes:
We think about this paper:
Volk, S. C. (2024). Assessing the Outputs, Outcomes, and Impacts of Science Communication: A Quantitative Content Analysis of 128 Science Communication Projects. Science Communication, 10755470241253858. https://journals.sagepub.com/doi/10.1177/10755470241253858
Will mentions this paper by C Thi. Nguyen:
Nguyen, C. T. (2021). The seductions of...2024-10-301h 14ReproducibiliTea PodcastS4E3 African Reproducibility Network (AREN) with Lamis Elkheir and Emmanuel BoakyeIn this episode, Will and Helena are joined by Emmanuel Boakye and Lamis Elkheir to share their experiences as scientists and Open Science advocates in the Global South and how they started the African Reproducibility Network (AREN).
African Reproducibility Network
Website: https://africanrn.org/
Twitter: https://twitter.com/africanrepro
Lamis Elkheir
LinkedIn: https://sd.linkedin.com/in/lamis-elkheir-b5844092
Twitter: https://twitter.com/lamiselkheir?lang=en
Emmanuel Boakye
LinkedIn: https://gh.linkedin.com/in/emmaboakye
Twitter: https://twitter.com/thescientistgh2024-10-1155 minReproducibiliTea PodcastS4E2 – Preprint Review with Jonny Coates from ASAPBioIn this episode, we welcome Queen Saikia as a host of the podcast! She and Will Ngiam are joined by Jonny Coates, Associate Director of ASAPBio, a non-profit organisation seeking to Accelerate Science and Publication in Biology. The topic of conversation is preprint review and peer review. Enjoy!
Show notes:
ASAPBio: https://asapbio.org/
PREreview: https://prereview.org/2024-09-1849 minReproducibiliTea PodcastS4E1: Reproducibility Training with Repro4Everyone with Nafisa Jadavji and Nele HaeltermanWe welcome back the ReproducibiliTea Podcast with Will and Helena chatting to Nafisa Jadavji and Nele Haelterman about Reproducibility for Everyone (R4E), a community-led initiative to run reproducibility workshops.
Show notes:
Repro4Everyone - https://repro4everyone.org2024-09-0648 min
Kannste Vergessen? – Der Podcast vom Lernen, Vergessen und ErinnernHirnforschen und Tee trinkenÜber Open Science, den ReproducibiliTEA Bochum und Transparenz in der Wissenschaft
Auch, wenn das Konzept schon älter ist: Ein echter Standard ist Open Science nicht überall. Der SFB 1280 Extinktionslernen hat sich selbst Regeln gegeben, nach denen die Wissenschaftler:innen hier offen arbeiten. Und unsere Gäst:innen Lianne Wolsink und Robert Reichert haben einen Journal Club gegründet, in dem sich Wissenschaftler:innen schon ab Beginn ihrer Karriere fortwährend treffen, um kritisch über Forschungspraktiken zu reflektieren, voneinander zu lernen, Fehler zu vermeiden und Techniken zu entwickeln, die ihre Wissenschaft wasserdicht machen: Das ist der ReproducibiliTEA der Ruhr-Uni Bochum, Teil ei...2024-03-0843 min
In Plain English: Science Made SimpleExploring the Future of Reproducible ScienceI chatted with William Ngiam of the ReproducibiliTEA podcast about scientific rigor and reproducibility, and the daunting, exciting, creative work of making science better.
If you liked this episode, check out Will & co. on the ReproducibiliTEA podcast!
2024 has already been hectic, but I hope to be back next week with more SfN Shorts!
In Plain English now has a Discord server! Join here to chat with other listeners, past experts, and guests about science, suggest new episode topics, and more!
Remember to follow In Plain English on Facebook , Twitter, and Instagram t...2024-01-091h 17
ReproducibiliTea PodcastS3E14: Season 3 CloserToday, Sarah, Will and Jan sit down to discuss the last season of ReproducibiliTea. We talk about Will's terrible terrible taste in puns, Jan's terrible taste in pizza, and Sarah's fall into FORRT(.org).
While we are wrapping up Season 3, stay tuned for a few more fun & exciting episodes we have planned for this year!
Not many footnotes this time:
Sarah mentioned sysmus: https://sites.google.com/view/sysmus/home
For more info go to ReproducibiliTea.org
For comments, questions, tips and tricks, and praise use our feedback form: forms.gle/H6jgUzbbpyauLxUC82023-09-151h 01
ReproducibiliTea PodcastS3E13: From Crisis To FORRTsitive ChangeToday, Will sits down with Max Korbmacher, Thomas Rhys Evans, and Flavio Azevedo, some of the authors of the paper "The replication crisis has led to positive structural, procedural, and community changes" to talk about the paper, FORRT, and Open Science communities.
Show notes:
The paper we discuss for this episode: Korbmacher, M., Azevedo, F., Pennington, C. R., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Nature Communications Psychology, 1(1), 3. https://www.nature.com/articles/s44271-023-00003-2
FORRT – The Framework for Open and Reproducible Research Training: https://fo...2023-09-0153 minReproducibiliTea PodcastS3E12: Revolutionizing Scientific Publishing with David ReinsteinToday, Will talks to David Reinstein about scientific publishing and The Unjournal.
The Unjournal: https://www.unjournal.org
The Unjournal on LinkedIn: https://www.linkedin.com/company/unjournal/
To get the latest updates: https://bit.ly/ujupdates
To apply for positions at the Unjournal: https://bit.ly/Ujwork
Will and David’s extended notes for the episode: https://docs.google.com/document/d/13rL6mq71GD6gPv5wBHr_lfX6YSKkLK9n6EAVfGhZhCk/edit?usp=sharing
For more info go to https://ReproducibiliTea.org
For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC82023-08-161h 07ReproducibiliTea PodcastS3E11: The Role Of Academic Societies with Björn Jorges and Sabrina Hansmann-RothToday, Will sits down with Björn Jorges and Sabrina Hansmann-Roth to discuss the role of academic societies in the science reform movement.
The poster session: https://www.visionsciences.org/2023-pre-data-collection-poster-session-satellite/
Korbmacher, M., Azevedo, F., Pennington, C., Hartmann, H., Pownall, M., Schmidt, K., ... & Evans, T. (2023). The replication crisis has led to positive structural, procedural, and community changes. Communications Psychology.
https://gala.gre.ac.uk/id/eprint/42730842730_EVANS_The_replication_crisis_has_led_to_positive_structural_procedural_and_community_changes.pdf
As always:
For more info go to ReproducibiliTea.org
For comments, questions, tips and tricks use our feedback form: forms.g...2023-07-2846 minReproducibiliTea PodcastS3E10: Registered Reports With Zoltan Dienes Live from SipsToday Sarah chats to Zoltan Dienes Live from SIPS!
Zoltan's keynote: https://www.youtube.com/watch?v=BxdGXLOC1Cc
Reviewing labor: https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-021-00118-2?ref=refind
Peer Community In Registered Reports: https://rr.peercommunityin.org/
Flourishing Science Think Tank paper: https://mindrxiv.org/4zrmd
There are a few little audio problems in this weeks episode, so special shoutout to our transcript, which is here: https://docs.google.com/document/d/1zd4o40QLn0o5q1Eyp966EoDdvu5XtT6O/edit?usp=sharing&ouid=115964780222242468834&rtpof=true&sd=true
For more info go to ReproducibiliTea.org
...2023-07-1455 minReproducibiliTea PodcastS3E9: Having Fun Isnt Hard When You Have A Library Card with Agata Bochynska and Matthew GoodToday, Sarah is joined by Agata Bochynska and Matthew Good from the University of Oslo's Open Research Team to talk about how vital libraries are to Open Science.
Links from this episode:
QualiFAIR: https://www.uv.uio.no/ils/english/about/organization/tlvlab/qualifair/
TIER2: https://tier2-project.eu/
Carpentries: https://carpentries.org/
ReproducibiliTea at UiO: https://www.ub.uio.no/english/libraries/dsc/open-repro-research/reproducibilitea/index.html
Preprint mentioned by Agata in the last segment: https://osf.io/kcvra/
Agata Twitter: https://twitter.com/AgataBochynska
Agata mastodon: https://fediscience.org/@agata
Digital Scholarship Centre website: https://www.ub.uio...2023-06-291h 08ReproducibiliTea PodcastS3E7: Invisible WorkloadToday, Sarah and Will discuss the invisible workload of making open science.
The paper on invisible workload: https://journal.trialanderror.org/pub/the-invisible-workload/release/1
The replication crisis has led to positive structural, procedural, and community changes: https://osf.io/r6cvx/
For more info go to ReproducibiliTea.org
For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC82023-06-2159 minReproducibiliTea PodcastS3E6: Registered Reports and Bayesian Fanclub with Nora SerresToday special guest Nora Serres talks with Sarah Sauve about Registered Report and how cool Bayesian statistics are.
Show notes:
Appetezer paper: Bayes factor design analysis: Planning for compelling evidence https://link.springer.com/article/10.3758/s13423-017-1230-y
Nora’s thesis: https://www.duo.uio.no/bitstream/handle/10852/95323/5/Master-thesis_NS.pdf
On registered reports:
https://www.cos.io/initiatives/registered-reports
https://osf.io/preprints/metaarxiv/x7aqr
https://www.nature.com/srep/journal-policies/registered-reports
PCI Registered Reports: https://rr.peercommunityin.org/PCIRegisteredReports/help/guide_for_authors
On Bayesian/statistics:
Bayesian Spectacles: https://www.bayesianspectacles.org/
Understanding Psychology as a Sc...2023-06-0951 minReproducibiliTea PodcastS3E5: ComputationalReproducibiliTeaToday, Will and Jan sit down with special guest and author of the paper "A large-scale study on research code quality and execution" Ana Trisovic to talk about the reproducibility of analysis code. Also windows.
Links from this episode:
Ana's paper: https://www.nature.com/articles/s41597-022-01143-6
Gratitude package: https://github.com/Pakillo/grateful
For more info go to ReproducibiliTea.org
For comments, questions, tips and tricks use our feedback form: forms.gle/H6jgUzbbpyauLxUC82023-05-2650 minReproducibiliTea PodcastS3E4: The UCU strikes wit Kat Almeida Warren and Ilse Pit! Important Update !
Since recording, UCU have announced that they will "ringfence £250,000 for members facing deductions for undertaking forms of ASOS, including the marking and assessment boycott." This is reassuring news for workers at universities where the VCs have threatened 50-100% pay deductions. But still no new offers from employers.
Today, Sophia and Jan are sitting down with @IlsePit(@fediscience.org) and @KatarinaWarren (k-almeidawarren.com) to chat about the ongoing UCU strikes in the UK, and how it is for ECRs to take industrial actions.
"The University and College Union (UCU) represents over 120,000 academics, lecturers, trainers, instructors, researchers, managers, administrators, computer s...2023-05-1250 minReproducibiliTea PodcastS3E3: Research Assessment Double FeatureWill, Helena and Jan talk about how we assess research and that it is weird we never really learned how to do Peer Review.
Papers we discussed:
Responsible Research Assessment Should Prioritize Theory Development and Testing Over Ticking Open Science Boxes: https://psyarxiv.com/ad74m/
A consensus-based tool for evaluating threats to the validity of empirical Research: https://psyarxiv.com/fc8v3
Will's blog post on peer review: https://williamngiam.github.io/blog/my_peer_review_process
The tool for checking p-Values: statcheck.io
The tool for checking mean values: https://journals.sagepub.com/doi/10.1177/1948550616673876
For more info go...2023-04-2848 minReproducibiliTea PodcastS3E02: Citational politics in open science: don't be a d*ck about itWe mentioned lots of resources this week, brace yourselves!
Books that changed our worlds
How To Read a Paper: The Basics of Evidence-Based Medicine by Trisha Greenhalgh
https://www.amazon.co.uk/How-Read-Paper-Evidence-Based-Medicine/dp/1444334360
How to Win Campaigns by Chris Rose
https://www.amazon.co.uk/How-Win-Campaigns-Steps-Success/dp/1853839620
Pollution is Colonialism by Max Liboiron
https://www.dukeupress.edu/pollution-is-colonialism
A Student’s Guide to Open Science by Charlotte Pennington
https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group
The Art of Statistics: Learning from Data by David Spiegelhalter
https://www.penguin.co.uk/books/294857/the-art-of-statistics-by-spiegelhalter-david/9780241258767
Tools we mention:
FORRT Academic Wheel of Pr...2023-04-141h 14ReproducibiliTea PodcastS03E01: Reform vs. Revolution (2... kinda)We are back!
And we brought... a whole list of reading recommendations!
Hosted and Produced by @Sarah_Sauve, @Will_Ngiam, and @VornhagenJB@hci.social
Edited by Jan Vornhagen
For more information visit ReproducibiliTea.org and send your feedback here: https://forms.gle/8nNLZ92YUcU1mGhc6. For a transcript, please refer to our youtube video: https://youtu.be/S7Ng6t34cSQ
This weeks tasty Tea-dbit/Appe-tea-zer:
Dienes' paper on democratic governance: https://royalsocietypublishing.org/doi/10.1098/rsos.220808
Things we talked about:
Dienes' paper on democratic governance: https://royalsocietypublishing.org/doi/10.1098/rsos.220808
Paul Feyerabend "Against Method"
Seductions of Clarity by C Thi...2023-03-311h 01
Research Culture Uncovered(S3 E2) Open research as an early career researcher and ReproducibiliTea with Kelly LloydIn our weekly Research Culture Uncovered conversations we are asking what is Research Culture and why does it matter? This episode is part of Season 3, hosted by Nick Sheppard who will be speaking to colleagues from both the University of Leeds and from other universities and organizations about open research, what it is, how it's practiced in different disciplines, and how it relates to research culture. In this episode Nick is joined by Kelly Lloyd. Kelly has recently completed an ESRC funded PhD in the Leeds Institute of Health Sciences on the topic of investigating decision making...2023-03-0844 minReproducibiliTea PodcastS02E12: "Live" from DagstuhlSophia and Jan serendipitously meet at a conference and immediately drag special guests @lonnibesancon, @lahariG, Wesley Willett into a room to talk about transparency, Open Science and Human-Computer Interaction.
For more Infos about the seminar visit https://www.dagstuhl.de/en/program/calendar/semhp/?semnr=22392
Or go to ReproducibiliTea.org
If you have questions or comments, throw them into the pot: forms.gle/HXHHvrNHn54MVHKb92022-10-1423 minReproducibiliTea PodcastS2E11: Open Peer Review of PublonsToday, Sam and Will have choice words about a website.
For more Infos go to ReproducibiliTea.org
If you have questions or comments, throw them into the pot: forms.gle/HXHHvrNHn54MVHKb9
Transcript: https://drive.google.com/file/d/1TL9lnW_HOL_BuN_wRDrWZb7e-kWUZedr/view?usp=sharing2022-09-0955 min
ReproducibiliTea PodcastS2E10: LimitsLinks from this Episode:
- Paper on the limits of open data for music science: https://emusicology.org/article/view/7646
- Feminist papers on open science: https://journals.sagepub.com/doi/pdf/10.1177/03616843211030926?casa_token=x6twMszrtQgAAAAA:7Wu2Z2V0X58Lo403dJPiLT3jL65YBErd4f6FE_bhcfnie_Sg4oqax9yaMF1R02WEsB4KKNvLu7b32xs
and
https://journals.sagepub.com/doi/full/10.1177/03616843211036460?casa_token=ZbT7O7sn04YAAAAA:gQ3xo-t3aEh38sIVEfs5SAMa_hJwz_iyDnjuM_q6J-7k8gLn-RuWxtuu-baSkGc1QcybOCFftdAncY0
- My Shiny App: https://sarahsauve.shinyapps.io/BrandAuditDashboard/
-Simulating LMEMs: https://shiny.psy.gla.ac.uk/lmem_sim/
-Preprint...2022-09-0248 minReproducibiliTea PodcastS2E09: Talking Power, Privilege and Community with Kohinoor DardaFind Kohinoor here: https://www.kohinoordarda.com/
Find all the stuff mentioned in this episode here:
The talk by a former senior admin, Dr. Max Liboiron: https://www.youtube.com/watch?v=Rya5Gom5o20
The book on the not-for-profit industrial complex: https://www.dukeupress.edu/the-revolution-will-not-be-funded
The organization Sarah is a part of and mentions in the episode, the Social Justice Cooperative of Newfoundland and Labrador: https://www.sjcnl.ca/
The Preprint on Open Science in India:
https://osf.io/preprints/aj9gw/
Open data sharing and the global south, who benefits?
https://www.science.org/doi/10.1126/science.aap8395
...2022-08-2652 minReproducibiliTea PodcastS2E08: CRediTWhat is the point of having a T in your name without being able to use it for a tea pun? ConTeabutorship was right there.
More information on CRediT here: https://credit.niso.org/
And more info on ReproducibiliTea here: https://ReproducibiliTea.org/
Questions? Answers? Comments? Feel free to drop them here: forms.gle/bRFcfiGQof43stoq6
Transcript: https://drive.google.com/file/d/1xi7sm5ExV7_sNejMIFUdH3CU7J4LWypH/view?usp=sharing2022-08-1938 minReproducibiliTea PodcastS2E07: Community ValuesToday, Sarah, Will, Sam and Jan discuss what kind of community we would like to foster and how you can get involved.
One of those ways, send us your questions and comments, in audio or text right here: https://forms.gle/bRFcfiGQof43stoq6
Another way: Find us on Twitter as @ReproducibiliT or send uns an E-Mail via ReproducibiliTea.org.
The latter is also the best place for all infos regarding how to start your own journal club and get involved!
Courtesy of Will, please find the transcript here:
https://drive.google.com/file/d/1C3P0oc0tf5gje2jUyahhNPvSdBRznREr...2022-08-1230 minReproducibiliTea PodcastS2E06: Slow ScienceSarah and Sam take a meandering discussion through slow science and what "slow" actually means in this context. Along the way, they also discuss some recommendations to promote slow science made in "Fast Lane to Slow Science" and how often scholarly critique often runs ahead of this slower pace.
Links and references:
Frith, U. (2020). Fast lane to slow science. Trends in Cognitive Sciences, 24(1), 1-2.
Pownall https://www.bps.org.uk/psychologist/slow-science-scholarly-critique
https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/
For more infos, visit ReproducibiliTea.org2022-08-0559 minReproducibiliTea PodcastS2E05: SIPS 2 - Improving Psychological Science BugalooSarah and Will went/logged on to this years SIPS respectively and talk about their conference experience. It was cool!
More infos www.improvingpsych.org and you can find the conference program here: https://docs.google.com/document/d/1Rka3f0ZY9BBZVIlZoVINrpyaJYeitpq1yKXBLukeMng/edit
For more information about us and how to start your own ReproducibiliTea Journal club go to ReproducibiliTea.org.
For a transcript check the Youtube version: https://youtu.be/FpW0n3fhx902022-07-2943 minReproducibiliTea PodcastS2E04: Conferences: Yay or Nay?Today, Sophia, Sam and Jan discuss conferences for the better part of an hour only to realize they is even still more to say... So stay tuned to this becoming like a regular episode.
Transcript will be added asap. Till then you can find subtitles in the youtube version of this episode:2022-07-2242 min
ReproducibiliTea PodcastS2E03: The Nowhere LabToday @Will_Ngiam talks to @eneayegba_ and @priyasilverst about the Nowhere Lab
Check out The Nowhere Lab at https://nowherelab.com/ or @theNowhereLab
This episode was produced by William Ngiam and edited by Jan Vornhagen.
Learn more on ReproducibiliTea.org
The Transcript will be added asap. Till then, you can find subtitles in the youtube version of this episode: https://www.youtube.com/channel/UCGbbo5KrnmAu_7zbnkm2swA2022-07-1542 minReproducibiliTea PodcastS2E02: Science Needs to Be BetterOur new co-hosts @Will_Ngiam, @Sarah_Sauve, and @VornhagenJB discuss how Science needs to be better while also sharing way to much cool ressources.
Transcript: https://drive.google.com/file/d/1lD0tTr-E-vv4lH8cJ9ItXedinORxnnQE/view?usp=sharing
Ressources like the blogpost that caused this:
https://williamngiam.github.io/Science_needs_to_be_better/
Cool stuff Sarah mentioned:
Indigenous Canada on Coursera: https://www.coursera.org/learn/indigenous-canada
An Indigenous Abolition Study Guide by the Yellowhead Institute: https://yellowheadinstitute.org/an-indigenous-abolitionist-study-guide/
Mapping accountability and relationships I got from Part 3 of Pollution is Colonialism by Max Liboiron: https://www.dukeupress...2022-07-0850 minReproducibiliTea PodcastS2E01: HandoverWe are back! ReproducibiliTea 2.0 not titled #TheNewNormal.
ReproducibiliTea is back with a new ambitions and rotating hosts, but first we hand the whole thing over.
For hopefully not the last time brewed by @OrbenAmy, @Sam_D_Parsons and @Cruwelli
Find the Transcript here: https://drive.google.com/file/d/1bTwTL6S2Ov9nlfSKdUxxEdweSM9Uwhmh/view?usp=sharing
For more info visit ReproducibiliTea.org2022-07-0118 min
Olinium8. Karel "Krápník" Berka: Od chemické olympiády k otevřeným chemickým datůmdoc. RNDr. Karel Berka, Ph.D. se z Mendelova gymnázia v Opavě, přes VŠ studium na PřF UK (fyzikální chemie a biochemie), doktorát na ÚOCHB dostal až na současnou pracovní pozici na fyzikální chemii na UP v Olomouci. V rozhovoru se bavíme nejen o zisku jeho přezdívky "Krápník", ale třeba také o tom, že nikdy neobhájil svou diplomku z biochemie a šel cestou teoretické chemie. Zajímavostí je, že s manželkou publikovali preprint (stal se virálem na soc. sítích) k předčasnému narození své druhé dcer...2020-11-2539 min
ReproducibiliTea PodcastEpisode 34 - Clarissa CarneiroEpisode 34 - Clarissa Carneiro
Early in May we talked to Clarissa (@clari_carneiro) from the Brazilian Reproducibility Initiative (@BrRepInitiative). This project is awesome - think many labs for Brazilian biomedical sciences, with a dash of meta in there too! Tune in to hear about this amazing project.
Links:
The project: https://www.reprodutibilidade.bio.br/home
The team: https://www.reprodutibilidade.bio.br/team
Partners: https://www.reprodutibilidade.bio.br/partners
Music credit: Kevin MacLeod - Funkeriffic
freepd.com/misc.php2020-06-1642 min
ReproducibiliTea PodcastEpisode 33 - Flavio Azevedo and building a FORRTEpisode 33 with Flavio Azevedo on FORRT
Our latest awesome ECR you need to watch out for is Flavio Azevedo @Flavio_Azevedo_. He was recently named one of one of the top 100 global shapers from Portugal! (https://www.100oportunidades.pt/) and by gosh he's just one of our favorite people too.
Flavio tells us all about the importance of openness in reproducibility in teaching. He is leading an amazing project: a Framework for Open and Reproducible Research Training, or FORRT. The project advocates for integrating open and reproducible research practices into not only methods training, but subject material too. FORRT will...2020-04-2155 min
ReproducibiliTea PodcastEpisode 32 - Crisis Research With Anne ScheelEpisode 32 - Research under Crisis with Anne Scheel
We have a very special guest in this lockdown episode of ReproducibiliTea: Anne Scheel (@annemscheel). We reflect on research during the COVID-19 pandemic and the wide range of responses from the research community.
We talked to Anne about her recent blogpost “Crisis research, fast and slow” (http://www.the100.ci/2020/03/26/crisis-research-fast-and-slow/) - her first in two years! A lot of research is being rushed to testing and to (pre)print; Anne worries that some of this goes against our collective efforts to promote ‘slow’ science.
Anne masterfully flips the interview on Amy and Sam...2020-04-071h 06
ReproducibiliTea PodcastEpisode 31 - Class 2 - What even is a replication?Episode 31: Class 2 - What even is a replication?
It’s week 2 of Amy’s “Psychology as a Robust Science” course and we are discussing replications. What are they? Is there a distinction between direct and conceptual replications? Do direct replications actually exist? Tune in to (maybe) find out!
Related papers and links
Open Science Collaboration (2015) Estimating the reproducibility of psychological science
https://science.sciencemag.org/content/349/6251/aac4716
Nosek and Errington (2017) Making sense of replications
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5245957/
Brian Nosek’s Queensland talk Amy mentioned (roughly between 5:00-15:00): https://www.youtube.com/watch?v=wsRmyW8GmJs
Gilbe...2020-03-1725 minReproducibiliTea PodcastSolidariTEA 2 - Green Advocacy With Sander van BreeSolidariTEA 2 - Green Advocacy With Sander van Bree
SolidariTEA is our mini series where we stand in solidarity against the worse side of academia, and stand in solidarity alongside the many awesome advocates for improving academia.
This time Sophia talks with Sander van Bree (@sandervanbree) about green advocacy. What actions can you take to reduce your negative impact on the environment; and what can academia do collectively? If you're still not sure on all things green advocacy after listening, hit him up on the twitters!
Sander is doubly awesome because he made Sam's life super easy by sharing these useful...2020-03-0327 min
ReproducibiliTea PodcastEpisode 30 - What Are We Doing? AKA The Golden PancakeEpisode 30 – What are we doing? AKA the golden pancake
We are starting a new 8 part series following the structure of Amy’s “Psychology as a Robust Science” course (https://www.amyorben.com/docs/syllabus.pdf). We will take a specific aspect from each week and dive into the detail.
This week: Introductions and Merton’s norms, what is Science? We discuss each of Mertons norms in turn; communalism, universalism, disinterestedness, and organised scepticism.
Useful resources:
Merton, Robert K. ‘The Normative Structure of Science’, 1942.
https://www.panarchy.org/merton/science.html.
this YouTube video for explanation: https://www.youtube.com/watch?v=00btFojQPiU...2020-02-1837 minReproducibiliTea PodcastEpisode 29 - Four Hero Origin Stories To Start 2020Episode 29 - Four Hero Origin Stories To Start 2020
Welcome to 2020. We are back to our regular recording and releasing schedule. Sorry for the wait, and thank you for your patience.
We begin with Sophia and Sam's starts to the year at the Advanced Methods for Reproducible Science workshop, aka #Repro20. All three of us have attended this workshop over the last 3 years. Last year Amy returned to talk about her year after #Repro18, and Sam returned to do the same about lessons from #Repro19.
The workshop schedule is here: https://osf.io/cdrfk/
And, slides from the talks and workshops...2020-02-0437 min
Everything Hertz100: Hundredth episode live special (with Daniel Lakens, Amy Orben, and Chris Chambers)To celebrate our 100th episode, which we video-streamed live, Dan and James were joined by three special guests: Daniel Lakens, Amy Orben, and Chris Chambers.
Here's what they covered in this episode:
James and Dan share their favourite episodes
The power of the Twitter direct message
Daniel Lakens joins us to discuss his recent work on helping people make better statistical decisions
Can you create cross-discipline effect size guidelines?
What would Jacob Cohen say if we could bring him back to life?
Academic backup career plans
Our new partnership with Prolific
James' piece on not treating...2020-01-271h 50Everything Hertz98: Episode titles are redundant, at best (with Sophia Crüwell)We chat with Sophia Crüwell (Meta-Research Innovation Center Berlin) about pre-registration and her recent work introducing pre-registration templates for cognitive modelling research.
Here's what we cover and some links:
Sophia’s PhD research
Sophia’s recent preprint: Preregistration in Complex Contexts: A Preregistration Template for the Application of Cognitive Models
The first version of the pre-print titled, preregistration is redundant, at best
The updated version of the preprint titled, "Is preregistration worthwhile?"
The Bayesian Spectacles blogpost on the first version of the pre-print
Data simulation from former guest Lisa DeBruine
The latest with reproducibilitea
The cargo...2019-12-1659 min
ReproducibiliTea PodcastEpisode 28 - Jade Pickering and Marta ToporEpisode 28 - Jade Pickering and Marta Topor
This time we are joined by the amazing Jade Pickering (@Jade_Pickering) and Marta Topor (@MartaTopor).
Jade and Marta share their own ReproducibiliTea experiences and an exciting new project on building tools for systematic reviews. Watch this space as the duo get ready to release this tool to the wilds!
Keep an eye out for the most positive and wholesome answers to our usual guest question "what advice do you have for other ECRs?"
Music credit: Be Jammin - Alexander Nakarada
freepd.com/world.php2019-12-1050 min
ReproducibiliTea PodcastSolidariTEA 1 - Robert StensonIn the first of our new SolidariTea sub-series, Amy Orben talks to Robert Stenson from the University of Nottingham, UK, where he is finishing a PhD at the School of Cultures, Languages and Area Studies. SolidariTea episodes are short podcast conversations with Early Career Researchers around the world that highlight injustices, campaigns, struggles and discussions mainly about working conditions, diversity and equality, employment, and work-life balance. Robert Stenson tells us about how his university has outsourced all grad student teachers to a temporary staffing company and how this has affected their working conditions and ability to strike (see his tweet...2019-12-0521 min
ReproducibiliTea PodcastReproducibillitea Episode 27 - PhD Advice Pt2Episode 27 - More advice on starting a PhD
We return to thinking about starting a PhD. This is part 2 of our series. Sophia brought Amy and Sam some more questions and they had thoughts. If you have any thoughts, comments, and different perspectives we would love to hear them. Please reach out and tell us your thoughts and experiences.
Question 4 - How much time did you spend on different PhD activities? e.g. writing papers, learning skills, reading, etc. AKA. work-work balance
Question 5 - How did you manage your supervisor and build a support team?
Question 6 - What was the...2019-11-1442 min
ReproducibiliTea PodcastEpisode 26 - Advice on starting a PhD part 1Episode 26 - Advice on starting a PhD
In a slight change of pace, Sophia interviews Amy and Sam for advice on starting a PhD. This is part 1 of a two part series on advice for new PhDs. If you have any thoughts or comments, hit up our twitter and join the conversation.
Caveat: Amy and Sam both completed their PhDs in Experimental Psychology at Oxford University. We did try to make any advice or comments as widely relevant as possible, however we should recognise the limits of our experience. We would love to hear from you, whether your institution/department...2019-10-2940 min
ReproducibiliTea PodcastStarting A ReproducibiliTea Journal Club With Jade Pickering And Angelika StefanEpisode 25 - Starting A ReproducibiliTea Journal Club With Jade Pickering And Angelika Stefan
Why did we start the ReproducibiliTea Journal club, and how can you start your own? Sam and Amy share their experiences before we listen in on a discussion with two very special guests. Jade Pickering (@Jade_Pickering) and Angelika Stefan (@ephemeralidea) discuss their experiences starting and running a ReproducibiliTea Journal club.
You can find ours, and others', ReproducibiliTea JC materials (including a starter pack) here: https://osf.io/3qrj6/
visit reproducibilitea.org for more information on starting your own JC, and all the info on the other...2019-10-1546 min
ReproducibiliTea PodcastEpisode 24 - ConferencesEpisode 24 - Conferences
This time we talk academic conferences! What do we thing about them as ECRs? Amy was at the ABCD workshop during recording (check out https://abcdstudy.org/) giving a hands-on picture of conference activities. Of course, we couldn't talk conferences without mentioning our favourite SIPS!(see https://improvingpsych.org/meetings/previous-sips-meetings/).
We compare conferences, discuss how we try to make the most out of our conference experience, and what conferences are like for ECRS
Music credit: Be Jammin - Alexander Nakarada2019-09-0358 min
ReproducibiliTea PodcastEpisode 23 -Nick FoxEpisode 23 - Nick Fox
Sophia and Amy have a great chat with Nick Fox (@NickFoxstats). Nick is a Research Scientist in the Center for Open Science.
Nick tells us his winding story from being a psych-hating undergrad, through biochem and behavioural neuroscience, to social psychology and meta-science (the full story is on his blog https://nickfox.netlify.com/post/trip-through-grad-school/).
Nick's thesis on "Scientist as Subject: How researcher behaviors influence psychological knowledge" as awesome, and you can read it here https://psyarxiv.com/6m7cn/
Nick gives an overview of SCORE - "Systematizing Confidence in Open Research and Evidence". The...2019-08-2059 min
ReproducibiliTea PodcastEpisode 22 - PsychbriefEpisode 22 - PsychBrief
This time we chat to another awesome ECR, PsychBrief! You may have seen his very popular blog on psychologocal methods http://psychbrief.com/ or seen him dropping truth bombs on twitter @PsyBrief
No, we don't reveal PsychBriefs real identity, but he does let us in on why he started and maintains his anonymous profile and his experiences in a not-so-open field. This was a great discussion, enjoy!
Music credit: Be Jammin - Alexander Nakarada
freepd.com/world.php2019-08-0639 min
ReproducibiliTea PodcastEpisode 21 - Katia DamerEpisode 21 - Katia Damer
This episode we talked to the co-founder and CEO of Prolific Academic (https://prolific.ac/), Katia Damer. Prolific Academic is a platform connecting researchers with a pool of research participants for online data collection. We discuss Katia's experience of founding Prolific as a start-up during her PhD and how prolific brings researchers and participants together.
Find Katia on twitter @ekadamer, and shutout to Prolific co-founder @Phelimb
Some highlights:
- Katia's journey to Prolific
- How we can use Prolific and whether it is the right tool for our research
- How participants can join
- Sam...2019-07-1747 min
ReproducibiliTea PodcastEpisode 20 - Priya SilversteinEpisode 20 - Priya Silverstein
Buckle up to hear Priya discuss running (and publishing) replication research and diversity in science. This was perhaps the most eventful recording session we have ever had, but we really enjoyed the conversation. The more diversity the better!
Music credit: Be Jammin - Alexander Nakarada
freepd.com/world.php2019-05-2845 minEverything Hertz84: A GPS in the Garden of Forking Paths (with Amy Orben)We chat with Amy Orben, who applies "multiverse" methodology to combat and expose analytical flexibility in her research area of the impact of digital technologies on psychological wellbeing. We also discuss ReproducibiliTea, an early career researcher-led journal club initiative she co-founded, which helps young researchers create local open science groups.
Here are some more details and links:
The tweet pointing our Dan's gramatical error in his usual introduction. THANKS DENIS
Is Twitter melting our brains?
The history of "new technology" panic
What's the next panic?
Moral entrepreneurs: profiting from moral panic
Specification curve analysis: a way...2019-05-2152 min
ReproducibiliTea PodcastEpisode 19 - Katie DraxEpisode 19 - Katie Drax
Introducing Katie Drax! an upcoming force in open and reproducible research!
Katie is leading MAPS: Mapping the Analytical PathS of a crowdsourced data analysis. Katie also leads the Bristol branch of the ReproducibiliTea Journal Club - with much more success in cataloging the JC than we manage. Katie also has some great advice for ECRs on their open science journey. We hope you enjoy listening as much as we did.
Katie's twitter: @katiedrax
MAPS: https://osf.io/9qke2/
ALSPAC study: http://www.bristol.ac.uk/alspac/
Music credit: Be Jammin - Alexander Nakarada
freepd.com/world...2019-05-1443 min
ReproducibiliTea PodcastEpisode 18 - Hannah HobsonEpisode 18 - Hannah Hobson
Meet Dr. Hannah Hobson, a true Registered Reports pioneer! Hannah was one of the early adopters of the registered report format, and during her PhD no less. We had to speak to her and hear her unique perspective on this research publication format.
Highlights and links:
Hannah's Mu registered report (https://www.sciencedirect.com/science/article/pii/S0010945216300570) (Sam highly recommends looking at the power analysis section as a good detailed example of how it's done)
The registered reports page on Centre for Open Science (https://cos.io/rr/)
Sarah Peters blogpost on why she took...2019-04-3041 min
ReproducibiliTea PodcastEpisode 17 - Lisa Spitzer And Tobias HeyckeEpisode 17 - Lisa Spitzer And Tobias Heycke
This week we have two special guests, two more awesome ECRs we want you to meet! Lisa Spitzer (@Lisa__Spitzer) and Tobias Heycke (@TobiasHeycke) discuss their project with us - using screen recordings of computerised experimental procedures to assist with reproducibility.
Check out Lisa and Tobias' preprint: https://psyarxiv.com/rbn8c/
In the interests of being doubly awesome, Lisa and Tobias have a separate tutorial on using Open Broadcaster Software (OBS) for the screen recording of your tasks: osf.io/3twe9
Music Credit: Be Jammin - Alexander Nakarada
freepd.com/world.php2019-04-1640 min
ReproducibiliTea PodcastEpisode 16 - Leonid TiokhinEpisode 16 - Leonid Tiokhin
In another SpecialiTEA we had the pleasure of chatting to Leo Tiokhin @LeonidTiokhin about his path to meta science and his current work.
We had a slight technical hiccup, but Sophia saved us with a back-up recording!
To find out more about Leo's recent Registered Report "An experimental test of the effects of competition forpriority on information sampling", check out the repository here https://osf.io/tn2vu/
Music Credit: Be Jammin - Alexander Nakarada
freepd.com/world.php2019-03-1943 min
ReproducibiliTea PodcastEpisode 15 - Responding To The Fallibility Of OthersEpisode 15 - Responding To The Fallibility Of Others
In this second part-er we revisit Dorothy Bishop's "Fallibility in Science: Responding to Errors in the Work of Oneself and Others". Last episode we discussed responding to one's own errors. We got so wrapped up that we didn't get onto responding to others errors. So, we're back;
Highlights:
How should we respond to other's errors
Oh my god this is complicated and has so many grey areas, including;
- Does career stage matter?
- Does the profile of the work matter?
- should we, and when should we, contact the authors to...2019-03-0636 min
ReproducibiliTea PodcastEpisode 14 - Responding to fallibility in scienceEpisode 14 - Responding to fallibility in science
This week we discuss Professor Dorothy Bishop's "Responding to fallibility in science" https://journals.sagepub.com/doi/10.1177/2515245918776632
Its been a while since we discussed a paper on the podcast. We really got into this discussion, and only got as far as "How should we deal with our own errors?". A whole two pages into the paper. We will revisit this paper soon to discuss responding to other's errors and distinguishing between the research and the researcher.
How do you think we should respond to our own errors? let us know!
Music credit:
Kevin...2019-02-1941 min
ReproducibiliTea PodcastEpisode 13 - Fighting The Impostor (syndrome)Episode 13 – Fighting the impostor (syndrome)
Hello 2019! Amy, Sophia, and Sam discuss impostor syndrome. While Sam was concerned this would leave him wanting to hide under the desk and cry-eat chocolate for the rest of the day, it turned into an unexpectedly uplifting talk.
Get in touch with your experiences; what helps? what makes it worse? how can we help each other?
Just a few links:
We should all feel a bit more like impostors – Julia Rohrer - http://www.the100.ci/2018/08/02/we-should-all-feel-a-bit-more-like-impostors/
Sam’s old, early blog post on what impostor syndrome feels like https://samdparsons.blogspot.com/2017/10/students-questions-4-what-is-imposter.html2019-01-2244 min
ReproducibiliTea PodcastEpisode 12 - Welcome Back Ivan FlisLast time we chatted to Ivan about his Thesis, but there was just too much to cover. Luckily, Ivan is a dude and agreed to chat to us again. Hope you enjoy
Ivan’s Dissertation “Discipline through method: Recent history and philosophy of scientific psychology (1950-2018)”
https://www.dropbox.com/s/amjv3oyu8u09nw2/2018%20thesis%20Ivan%20Flis%20final%20version.pdf?dl=0
Hornstein, G. A. (1988). Quantifying Psychological Phenomena: Debates, Dilemmas, and Implications. In J. G. Morawski (Ed.), The Rise of Experimentation in American Psychology (pp. 1–34). New Haven, CT: Yale University Press. (this whole book is a classic on the history of exper...2018-12-1148 min
ReproducibiliTea PodcastEpisode 11 - Ivan FlisEpisode 11 - Ivan Flis
This week we had an awesome chat with the super-interdisciplinary Ivan Flis. Ivan is like a cake made from psychology, history, and philosophy of science. This cake was the perfect compliment to our Repro-Tea. There was so much to cover that we will have Ivan back next time too!
Just a few of the amazing resources Ivan mentioned:
Deborah Mayo’s book, Statistical Inference as Severe Testing: How to Get Beyond the Statistics Words, Amy still doesn’t understand what she is really reading though…. (Sam is doing no better) (https://www.cambridge.org/core/books/statis...2018-11-2752 min
ReproducibiliTea PodcastEpisode 10 - Leaving Academia (part 2)In a previous episode we discussed leaving academia. Lots of people who had left academia responded to our survey and gave us some insight into why they left and what they are doing now. This time we dive into the advice they offered to ECRs given the likely event of leaving academia. Thank you to everyone that responded, and for your advice.
We will post to this list as we find more resources :)
Music credit: Be Jammin – Alexander Nakarada
freepd.com/world.php2018-11-1345 min
ReproducibiliTea PodcastEpisode 9 - Did you know we love the open science community?Episode 9 - Did you know we love the open science community?
In this episode we tackle how the open science community is perceived, and our own experiences as members of it. We talk (in a non-specific way) about recent and recurring discussions concerning public critique and how harassment is dealt with on social media. We are happy to continue this discussion; if we have missed anything, or could have been more sensitive to specific issues let us know, we are interested in continuing the conversation. We love the open science community generally and want to reassure those unsure about open...2018-10-3052 min
ReproducibiliTea PodcastSpecialiTEA 5 - PederSpecialiTea episode 5 – Peder Isager
We get talking to Peder Isager about his PhD work, equivalence testing, and the psychological science accelerator. You can find Peder on twitter (@peder_isager); he also has a website https://pedermisager.netlify.com/
Some Equivalence testing resources:
“Equivalence Testing for Psychological Research: A Tutorial” from Lakens, Scheel, and Isager http://journals.sagepub.com/doi/full/10.1177/2515245918770963
“TOST equivalence testing R package (TOSTER) and spreadsheet” http://daniellakens.blogspot.com/2016/12/tost-equivalence-testing-r-package.html
The psychological science accelerator:
The website https://psysciacc.org/
Twitter @PsySciAcc
Pangea – Jake Westfall: the paper (http://jakewestfall.org/publications/pangea.pdf), and the shiny app (https://jake...2018-10-161h 00
ReproducibiliTea PodcastSpecialiTea 4 - Leaving Academia (part 1)SpecialiTea Episode 4 – Leaving academia (part 1)
A few weeks ago we put out a survey to get a picture of peoples’ reasons for leaving academia and what advice they have for ECRs. We had lots of responses and dive into them here. There wasn’t quite enough time to really dig into the advice offered. But, that’s where you lovely people can come in. Our survey is still open (https://docs.google.com/forms/d/e/1FAIpQLSdad9GpUbqKebCxUSbUJXVvyOWH3gtC7zO77trFIo-SwUh-og/viewform), so if you have any advice to offer let us know there or DM us on twitter @ReproducibiliT
Some sho...2018-10-0247 min
ReproducibiliTea PodcastEpisode 8 - The preregistration revolutionEpisode 8 - The preregistration revolution
It’s preregistration time! Distinguishing confirmatory (or prediction) and exploratory (or postdiction) analyses. We discuss this awesome paper from Nosek, Ebersole, DeHaven, and Mellor (2018) “The preregistration revolution”. Find the paper here http://www.pnas.org/content/early/2018/03/08/1708274114
Highlights
[1:00] What is a preregistration anyhow?
[2:00] Prediction vs postdiction
[3:00] Why is preregistration important? Combatting bias; distinguishing confirmatory and exploratory analyses.
[4:45] In a perfect world, do we need preregistration?
[10:00] What else does preregistration guard against, or not?
[12:00] Getting critical; does prereg always work? Probably not, we need quality control. So, is prereg better than nothing? maybe not…
[14:30] Where are your pre...2018-09-1849 min
ReproducibiliTea PodcastEpisode 7 - Natural Selection Of Bad ScienceThis week we talk about “The natural selection of bad science” from Paul Smaldino and Richard McElreath. We get bummed, but try to put a positive spin on things. You can find the paper here http://rsos.royalsocietypublishing.org/content/3/9/160384
Highlights
[1:00] “The natural selection of bad science” http://rsos.royalsocietypublishing.org/content/3/9/160384
[5:00] the need to distinguish yourself as a scientist - innovative and/or high output?
[6:45] Our next SpecialiTEA - we want to hear your experiences about leaving academia
[11:00] “when a measure becomes a target it ceases to be a good measure”
[13:30] could quality control help ‘fix’ metrics?
[15:00] Competition - how quickly can...2018-09-0458 min
ReproducibiliTea PodcastSpecialiTEA 3 - Julia RohrerSpecialiTea 3: Julia Rohrer
We talk to Julia Rohrer (@dingding_peng)about her blog, skills training as a PhD and what doesn’t get talked about enough in the Open Science Community. You can find her amazing blog here: http://www.the100.ci, where she is joined by Anne Scheel (@annemscheel), Ruben Arslan (@rubenarslan) and Malte Elson (@maltoesermalte).
We want to note that this podcast had the world’s most WEIRDEST Skype setup as we forgot our headphone splitters. This meant that both Amy and Sophia were using separate headphones and were on muted Skype calls to each other while Sam was...2018-08-2838 min
ReproducibiliTea PodcastEpisode 6 - Open ScienceThis week we talked about “Open Science: What, Why, and How” from Spellman, Gilbert, and Corker. You can find the paper on OSF https://osf.io/gv6r4/
0:14 Sophia is leaving Oxford :( (But the Podcast will continue)
0:48 This week’s focus: Open Science What Why and How; few in the JC read it but the discussion was awesome https://osf.io/gv6r4/
2:15 Shoutout to Matt Jaquiery @MJaquiery
2:40 Broad Meaning of Open Science -- what do we mean; potentially misleading to include things beyond Open?
5:10 The Centre for Science that’s Actually Science
6:05 Focussing on Open Data as “open signalling”
7:30 What even is...2018-08-2151 min
ReproducibiliTea PodcastEpisode 5 - Is The Crisis OverblownEpisode 5 - Is the replicability crisis overblown?
This week talked about Pashier & Harris’s (2012) “Is the replicability crisis overblown? Three arguments examined.” http://journals.sagepub.com/doi/10.1177/1745691612463401
Highlights:
[1:00] Reproducible, replicable, robust, or generalisable? Table taken from Kirstie Whitaker, shared by @RemiGau http://sx.doi.org/10.6084/m9.figshare.4244996
[2:00] Argument 1: False Positive Rate
[3:00] False Positive Psychology - http://journals.sagepub.com/doi/abs/10.1177/0956797611417632
(which we mention more about in Episode 2)
[4:45] Dorothy Bishop on data simulation - http://deevybee.blogspot.com/2013/06/interpreting-unexpected-significant.html
[7:15] AMPPS paper Amy mentions - The Prior Odds of Testing a True Effect in Cognitive and Social Psychology
http://journals.sagepu...2018-08-1443 min
ReproducibiliTea PodcastSpecialiTEA 2 - SIPSSophia and Sam headed to this year's Society for the Improvement of Psychological Science (SIPS) meeting. We got talking to some super interesting and awesome researchers (follow them on twitter, handles below) to share the experience with all of you!
Check out https://osf.io/ck28s/ for all things taught and created at SIPS 2018
Highlights:
[0:00] What is SIPS anyway?
[8:00] Elizabeth Page-Gould (@page_gould) and Michael Inzlict (minzlicht) - Creating and finding open science jobs
[15:30] Kristen Lane (@kristen_a_lane) and Heather Urry (@HeatherUrry) - Teaching reproducible research
[19:30] Jessica Flake (@JkayFlake) and Eiko Fried (@EikoFried)
[26:00] Michele Nuijten (@MicheleNuijten), Nick Brown...2018-08-0750 min
ReproducibiliTea PodcastEpisode 4 - Reproducibility nowEpisode 4 - Reproducibility Now
This week we dive into the Open Science Collaboration’s (2015) paper “Estimating the reproducibility of psychological science”
http://science.sciencemag.org/content/349/6251/aac4716
Highlights:
[1:00] This paper has all of the authors
[1:30] Direct vs conceptual replications
[4:30] PhD students running replications as the basis of extending a paradigm
[6:00] The 100 studies paper methods in brief
[8:00] everything’s available for this collaborative effort, and that is awesome
(https://osf.io/ezcuj)
[9:00] Reproducibility vs Replicability - what are we actually talking about
[9:30] Oxford summer school in reproducibiltiy
(https://www.eventbrite.co.uk/e/oxford-reproducibility-school-tickets-48405892327)
[11:00] paper discussing the computational reproducibility of papers
[15:00] Replicat...2018-07-3142 min
ReproducibiliTea PodcastEpisode 3 - Questionable Research PracticesEpisode 3 - Questionable Research Practices
This week we discuss the prevalence of questionable research practices in John, Lowenstein, & Prelec “Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling” http://journals.sagepub.com/doi/abs/10.1177/0956797611430953
Highlights:
[2:00] What are questionable research practices?
[3:00] QRP as a political term
[6:00] a pinch of transparency
[8:00] QRPs as a grey area?
[12:00] Does defining a QRP as ‘just’ questionable water down the message of ‘this is wrong’?
[14:00] Do John, Lowenstein, & Prelec practice what they preach in this paper? We discuss the methodology
[19:30] Figure 1 - What do we see? “Everybody’s doing it”
[22:00] Passing on QRPs throughout the (re...2018-07-2437 min
ReproducibiliTea PodcastSpecialiTEA 1 - Tim Van Der ZeeSpecialiTEA 1 – Tim van der Zee
In our first SpecialiTEA episode we talk to Tim van der Zee (@research_Tim) about error detection, navigating critique on social media, and the positive side of the replication crisis for early career researchers.
Highlights
[1:30] Hey Tim! What are you working on?
[2:30] Tim: I am meta-science BATMAN
[3:30] Error detection: Impossible numbers in papers
[5:00] Tim’s favourite errors: we’re not naming any names, but Tim will talk about some food studies…
[10:00] can you trust the results If the basic numbers are wrong?
[12:00] But I still believe these findings…
[15:00] critiques against error detection
[18:00] Tim’s twitter feed (@resea...2018-07-1732 min
ReproducibiliTea PodcastEpisode 2 - Examining analytic flexibilityEpisode 2 - Examining analytic flexibility
This week discuss analytic flexibility in “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant” from Simmons, Nelson, & Simonsohn (2011) http://journals.sagepub.com/doi/abs/10.1177/0956797611417632
Highlights:
[1:00] What is a false positive anyway?
[3.30] Dead salmon fMRI study http://prefrontal.org/files/posters/Bennett-Salmon-2009.pdf
[4:30] Pregnant men as our example of false positives
[8:30] Garden of forking paths
[11:00] Gelman & Lokens (2013) http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
[18:00] precognition - we needed results that broke physics to show us the problems in psychology
[21:00] Table 1 and simulating researcher degrees of freedom
[24:30] “Our go...2018-07-1038 min
ReproducibiliTea PodcastEpisode 1 - The problem definedIn our first journal club podcast we discuss "A manifesto for reproducible science" from Nosek et al. www.nature.com/articles/s41562-016-0021
Highlights
[0:00] Sam butchers everybody's names, right from the start
[7:00] "The first principle is that you must not fool yourself — and you are the easiest person to fool" - Richard Feynman
[9:00] Figure 1; the best figure; the every-figure
[13:30] HARKing (Hypothesising After the Results are Known), distinguishing confirmatory and exploratory analyses, preregistration
[17:00] Power and false positives
[19:00] Button et al. "Power failure: why small sample size undermines the reliability of neuroscience" https://www.nature.com/articles/nrn3475
[26:00] storytelling
[30:00] The garden of...2018-07-0346 minEverything Hertz64: Salami slicingDan and James talk about the recent SIPS conference answer a listener question on "salami slicing" the outcomes from one study into multiple papers.
Here's what they cover:
What is the SIPS conference? [0:24]
A SIPS proposal for Google scholar to highlight commentaries and replication attempts on specific articles [15:42]
James and Dan’s favourite Hertz episodes [20:43]
We answer a listener question on Salami slicing [28:45]
Can you publish too much? [48:10]
Links
SIPS conference: https://www.improvingpsych.org/SIPS2018/
Reproducibilitea podcast: https://soundcloud.com/reproducibilitea
Salami slicing tweet: https://twitter.com/academicswrite/status/1008719899940786176
Cumulative impact factors: http://kh...2018-07-021h 01
ReproducibiliTea PodcastEpisode Zero - IntroductionsAmy, Sophia, and Sam introduce the ReproducibiliTea Journal Club Podcast - where we serve up mugs of ReproducibiliTea in blends of transparency, openness, and robustness with a spoonful of science.
In upcoming episodes we will cover (the pdfs can also be found at https://drive.google.com/drive/folders/1dpJ7LBLjXIBhiWBJMlG-B-UgziJxyS7D);
1. A manifesto for reproducible science https://www.nature.com/articles/s41562-016-0021
2. False-Positive psychology http://journals.sagepub.com/doi/abs/10.1177/0956797611417632
3. Measuring the prevalence of questionable research practices with incentives for truth telling http://journals.sagepub.com/doi/abs/10.1177/0956797611430953
4. Estimating the Reproducibility of Psychological Science http://science.sciencemag...2018-06-2724 min