ReproHack @ Leiden University

A hands-on Reproducibility Hackathon


📅 30th November, 2019
⏰ 10 am - 5pm
📍 Leiden University Library

Welcome 👋

What’s a ReproHack?

We are all excited by the progress made by many authors to make their papers reproducible by publishing associated code and data.

We know how challenging it can be so we want to showcase the value of the practice, both for original authors and as a learning experience for those who attempt to reproduce the work.

During a ReproHack, participants attempt to reproduce published research of their choice from a list of proposed papers with publicly available associated code and data. Participants get to work with other people’s material in a low pressure environment and record their experiences on a number of key aspects, including reproducibility, transparency and reusability of materials. At the end of the day we regroup, share our experiences and give feedback to the authors.


It’s imperative to note that ReproHacks are by no means an attempt to criticise or discredit work. We see reproduction as beneficial scientific activity in itself, with useful outcomes for authors and valuable learning experiences for the participants and the research community as a whole.

We strive to make this event open and inclusive to all. As such we ask you to read our Code of Conduct. By participating, you are expected to uphold this code.


Join the event and reproduce ♻

Register and save your spot by clicking here!

Join us at the ReproHack and get working with other people’s material.

Benefits to participants:

  • Practical experience in reproducibility with real published materials and the opportunity to explore different tools and strategies.

  • Inspiration from working with other people’s code and data.

  • An appreciation that reproducibility is non trivial but that opening up your work for more people to engage with is the best way to help improve it. An appreciation that reproducibility has community value beyond just the validation of the results. For example, access to such materials increases the potential for reuse and understanding of the work.

Benefits to the whole research community:

  • Assessment of how reproducible papers are ‘out of the box’.

  • Evaluation of how successful current practices are and for what purpose.

  • Identification of what works and where the most pressing weaknesses in our approaches are.



Register


Propose a paper 📜

Click here to nominate your paper!

You’ve put a lot of effort into making your work reproducible. Now let people learn from and engage with it!

We invite nominations for papers that have both associated code and data publicly available. We also encourage analyses based on open source tools as we cannot guarantee participants will have access to specialised licenced software.


Benefits to authors:

  • Feedback on the reproducibility of your work.

  • Appreciation for your efforts in making your work reproducible.

  • Opportunity to engage others with your research.


Nominate Paper
Proposed papers:


1. Supercurrent-induced Majorana bound states in a planar geometry

Supercurrent-induced Majorana bound states in a planar geometry. André Melo, Sebastian Rubbert, Anton R. Akhmerov. 2019. SciPost Phys. 7, 039.

submitted by André Melo twitter

Why should we attempt to reproduce this paper?

We’ve tried to make it as easy as possible to reproduce. There’s some fun physics on the paper and it’s all done with Python!

Paper URL: https://dx.doi.org/10.21468/SciPostPhys.7.3.039

Data URL: https://doi.org/10.5281/zenodo.2653483

Code URL: https://doi.org/10.5281/zenodo.2653483

Useful programming skills: Python

2. Social-evaluative threat: Stress response stages and influences of biological sex and neuroticism

Poppelaars, E. S., Klackl, J., Pletzer, B., Wilhelm, F. H., & Jonas, E. (2019b). Social-evaluative threat: Stress response stages and influences of biological sex and neuroticism. Psychoneuroendocrinology, 109, 104378. https://doi.org/10.1016/j.psyneuen.2019.104378

submitted by Eefje S. Poppelaars twitter github

Why should we attempt to reproduce this paper?

This is a small dataset with a lot of missing data, so it’s quite challenging to produce reliable results. It uses multiple imputation to fill the missing data, so it would be interesting to see whether the results hold up when this is redone. However, since the multiple imputation takes a couple of hours to run (on a decent laptop), the final multiply imputed data is also included. Additionally, multiply imputed data needs a different statistical analysis approach, which you can get familiar with.

Paper URL: https://doi.org/10.1016/j.psyneuen.2019.104378

Data URL: https://doi.org/10.1016/j.dib.2019.104645

Code URL: http://dx.doi.org/10.17632/7vj8r76s6f.10

Useful programming skills: R

3. Impact of predictor measurement heterogeneity across settings on the performance of prediction models: A measurement error perspective

Luijken, K., Groenwold, R. H., Van Calster, B., Steyerberg, E. W., & van Smeden, M. (2019). Impact of predictor measurement heterogeneity across settings on the performance of prediction models: A measurement error perspective. Statistics in medicine.

submitted by Kim Luijken github

Why should we attempt to reproduce this paper?

This methodological paper contains explanations and recommendations based on simulation studies. It is my first attempt at making a paper reproducible. I would be interested in knowing whether reproduced results are equal (or similar enough) and whether the code is user friendly.

Paper URL: https://onlinelibrary.wiley.com/doi/full/10.1002/sim.8183

Data URL: https://github.com/KLuijken/Prediction_Measurement_Heterogeneity_Predictor

Code URL: https://github.com/KLuijken/Prediction_Measurement_Heterogeneity_Predictor

Useful programming skills: R

4. Improving on Adjusted R-Squared

Karch, J. (2019, September 16). Improving on Adjusted R-Squared. https://doi.org/10.31234/osf.io/v8dz5

submitted by Julian D. Karch

Why should we attempt to reproduce this paper?

First, I think I came quite far in making it reproducible. The paper uses a simulation study. All the steps from the raw results of the simulation study to the final manuscript can be reproduced by clicking one button. This works because all code + all dependencies are stored online within a virtual machine that anybody can access. So, I think it is a quite good example for others to learn from. Second, the code running the simulation study itself is not included in this process because it would take far too long to run on one machine. I ran the code on a supercomputer. I would be interested in how people would try to reproduce such long-running code and whether they have feedback on how to improve sharing such long-running code.

Paper URL: https://doi.org/10.31234/osf.io/v8dz5

Data URL: https://doi.org/10.24433/CO.8023088.v1

Code URL: https://doi.org/10.24433/CO.8023088.v1

Useful programming skills: R

5. Spectral measure of color variation of black-orange-black (BOB) pattern in small parasitoid wasps (Hymenoptera: Scelionidae), a statistical approach

NA

submitted by Marcela Alfaro CĂłrdoba twitter github

Why should we attempt to reproduce this paper?

PLOS one ensures that sharing code and data is enough for reproducibility. I think is not, and I want your opinion on giving them more specific recommendations based on what I did wrong in our article.

Paper URL: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6812806/#!po=0.467290

Data URL: https://github.com/malfaro2/Mora_et_al

Code URL: https://github.com/malfaro2/Mora_et_al

Useful programming skills: R

6. Use of significance test logic by scientists in a novel reasoning task

Morey and Hoekstra (in preparation)

submitted by Richard Morey twitter github

Why should we attempt to reproduce this paper?

The format of the paper is a bit unusual: it is contained, and compiled as, an R package. Although this would seem, on its face, to make it easier to reproduce, it is an open question how obvious it will be. I wonder to what extent people reproducing the results would prefer this to simple R scripts.

Paper URL: https://richarddmorey.github.io/Morey_Hoekstra_StatCognition/extras/manuscript.pdf

Data URL: https://github.com/richarddmorey/Morey_Hoekstra_StatCognition

Code URL: https://github.com/richarddmorey/Morey_Hoekstra_StatCognition

Useful programming skills: R

7. Cell Contractility Facilitates Alignment of Cells and Tissues to Static Uniaxial Stretch

Rens, E. G., & Merks, R. M. H. (2017). Cell Contractility Facilitates Alignment of Cells and Tissues to Static Uniaxial Stretch. Biophysical Journal, 112(4), 755–766.

submitted by Roeland Merks twitter github

Why should we attempt to reproduce this paper?

This paper shows a fun and interesting simulation result. I find it (of course) very important that our results are reproducible. In this paper, however, we did not include the exact code for these specific simulations, but the results should be reproducible using the code of our previous paper in PLOS Computational Biology (Van Oers, Rens et al. https://doi.org/10.1371/journal.pcbi.1003774). I am genuinely curious to see if there is sufficient information for the Biophys J paper or if we should have done better.

Other people have already successfully built upon the 2014 (PLOS) paper using our code; see e.g., https://journals.aps.org/pre/abstract/10.1103/PhysRevE.97.012408 and https://doi.org/10.1101/701037).

Paper URL: http://doi.org/10.1016/j.bpj.2016.12.012

Data URL: NA

Code URL: https://doi.org/10.1371/journal.pcbi.1003774.s006

Useful programming skills: C and Matlab for visualization

8. Determination of the fundamental absorption and optical bandgap of dielectric thin films from single optical transmittance measurements

A. Tejada, L. Montañez, C. Torres, P. Llontop, L. Flores-Escalante, F. De Zela, A. Winnacker, and J. A. Guerra, Appl. Opt, doc. ID 377072 (posted 11 November 2019, in press).

submitted by Jorge Andres Guerra

Why should we attempt to reproduce this paper?

We propose a simple method to retrieve optical constants from single optical transmittance measurements, in particular in the fundamental absorption region. The construction of needed envelopes is arbitrary and will depend on the user. However, the method should still be robust and deliver similar results.

Paper URL: https://www.osapublishing.org/ao/upcoming_pdf.cfm?id=377072

Data URL: https://www.dropbox.com/s/1i8yvbomufytetj/Tsim%20Ts%20%20n%20and%20k.rar?dl=0

Code URL: NA

Useful programming skills: Python and/or Mathematica

9. Does honesty require time? Two preregistered replications of Experiment 2 of Shalvi, Eldar, and Bereby-Meyer (2012)

Does honesty require time? Two preregistered replications of Experiment 2 of Shalvi, Eldar, and Bereby-Meyer (2012). Preprint. retrieved from https://osf.io/qbk6z/

submitted by Bruno Verschuere twitter

Why should we attempt to reproduce this paper?

We aimed at making the analyses fully reproducible and are curious to see whether we succeeded in that aim.

Paper URL: https://osf.io/qbk6z/

Data URL: https://osf.io/fnh9u/

Code URL: https://osf.io/fnh9u/

Useful programming skills: R

10. Who is more likely to influence others? A value-based approach to pro-environmental social influence behavior (Preprint)

Kis, A., Verschoor, M., & Sargisson, R. (2019, November 18). Who is more likely to influence others? A value-based approach to pro-environmental social influence behavior (Preprint). https://doi.org/10.31234/osf.io/f6qut

submitted by Andrea Kis

Why should we attempt to reproduce this paper?

As psychology is currently facing a reproducibility crisis, it is increasingly important for research psychologists to team up with such initiatives and try to reproduce their analyses. This paper deals with a medium-sized dataset of 193 participants, with some missing data. Because we used multiple imputations to fill in the missing data, reproduction could shed some light on whether the analysis stands even if reproduced. Within the data files you can find the original data (Data.xlsx) and the five times imputed data (Data Imputed.sav), so you can choose to work with either version. Please note that in the original data, input variables are scaled 1-7, hence values above 7 (implicating “don’t know” answers) should also be coded as missing data (see Data Codebook.pdf for codes).

Paper URL: https://psyarxiv.com/f6qut/

Data URL: https://osf.io/fawy2/

Code URL: https://osf.io/z2awp/

Useful programming skills: R

11. Explicit (but not implicit) environmentalist identity predicts pro-environmental behavior and policy preferences

Brick, C., & Lai, C. K. (2018). Explicit (but not implicit) environmentalist identity predicts pro-environmental action. Journal of Environmental Psychology, 58, 8-17. doi.org/10.1016/j.jenvp.2018.07.003

submitted by Cameron Brick twitter

Why should we attempt to reproduce this paper?

The results of the individual studies (4) could be interpreted in support for the hypothesis, but the meta-analysis suggested that implicit identification was not a useful predictor overall. This conclusion is an important goalpost for future work.

Paper URL: https://www.sciencedirect.com/science/article/abs/pii/S0272494418300549?via%3Dihub

Data URL: https://osf.io/c2bf8/

Code URL: https://osf.io/c2bf8/

Useful programming skills: Mini-meta-analysis, R, Stata

12. Manuscripts, Metadata, and Medieval Multilingualism: Using a Manuscript Dataset to Analyze Language Use and Distribution in Medieval England

Murchison, Krista A., and Ben Companjen. “Manuscripts, Metadata, and Medieval Multilingualism: Using a Manuscript Dataset to Analyze Language Use and Distribution in Medieval England.” DH Benelux Journal 1 (2019).

submitted by Ben Companjen github

Why should we attempt to reproduce this paper?

We would like to hear from others whether the process is repeatable from the paper and code documentation. Also, this is a paper from the Humanities :) !

Paper URL: http://journal.dhbenelux.org/journal/issues/001/Article-Murchison/murchison-main.tex.html

Data URL: https://doi.org/10.17026/dans-zxr-juar

Code URL: https://doi.org/10.5281/zenodo.1472267

Useful programming skills: Python

13. Quantitative analysis of spectroscopic Low Energy Electron Microscopy data: High-dynamic range imaging, drift correction and cluster analysis

Jong T.A. de, Kok, D.N.L, Torren A.J.H. van der, Schopmans H., Tromp R.M., Molen S.J. van der & Jobst J. (2019), Quantitative analysis of spectroscopic Low Energy Electron Microscopy data: High-dynamic range imaging, drift correction and cluster analysis, Ultramicroscopy .

submitted by Tobias A de Jong twitter github

Why should we attempt to reproduce this paper?

Low Energy Electron Microscopy (LEEM) is a somewhat specific form of electron microscopy used to study surfaces and 2D materials. In this paper we describe a set of data processing techniques applied to LEEM and adapted to the peculiarities of LEEM. This is combined with a parallelized Python implementation using Dask in separate notebooks. So if you are interested in microscopy, image analysis, clustering of experimental physics data or parallel Python, this paper should be interesting to you.

Paper URL: https://doi.org/10.1016/j.ultramic.2019.112913

Data URL: https://doi.org/10.4121/uuid:7f672638-66f6-4ec3-a16c-34181cc45202

Code URL: https://doi.org/10.5281/zenodo.3539538

Useful programming skills: Python

14. Algorithm configuration data mining for CMA evolution strategies

Sander van Rijn, Hao Wang, Bas van Stein, and Thomas BĂ€ck. 2017. Algorithm configuration data mining for CMA evolution strategies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17). ACM, New York, NY, USA, 737-744. DOI: https://doi.org/10.1145/3071178.3071205

submitted by Sander van Rijn twitter github

Why should we attempt to reproduce this paper?

The original data took quite a while to produce for a previous paper, but for this paper, all tables and figures should be exactly reproducible by simply running the jupyter notebook.

Paper URL: https://dl.acm.org/citation.cfm?id=3071205

Data URL: https://github.com/sjvrijn/cma-es-configuration-data-mining/tree/master/arff

Code URL: https://github.com/sjvrijn/cma-es-configuration-data-mining

Useful programming skills: Python

15. Don’t Hold My Data Hostage - A Case For Client Protocol Redesign

Mark Raasveldt and Hannes MĂŒhleisen: Don’t Hold My Data Hostage - A Case For Client Protocol Redesign, 43rd International Conference on Very Large Data Bases (VLDB2017)

submitted by Hannes MĂŒhleisen twitter github

Why should we attempt to reproduce this paper?

We made a huge effort to ensure the paper is reproducible. But is it?

Paper URL: https://hannes.muehleisen.org/p852-muehleisen.pdf

Data URL: https://zenodo.org/record/1305845

Code URL: https://github.com/Mytherin/Protocol-Benchmarks

Useful programming skills: Python

16. Spatial modelling of rice yield losses in Tanzania due to bacterial leaf blight and leaf blast in a changing climate

Spatial modelling of rice yield losses in Tanzania due to bacterial leaf blight and leaf blast in a changing climate. C. Duku, A. H. Sparks, S. J. Zwart. Climatic Change 135.3-4 (2016) pp. 569–583. Springer Nature. doi: 10.1007/s10584-015-1580-2

submitted by Adam Sparks twitter github

Why should we attempt to reproduce this paper?

This was my third attempt at making a paper fully reproducible. To date I it’s the most reproducible that I have published. I’m interested to know what stumbling blocks exist that I’m not aware of (aside from needing software like ArcGIS to fully rerun the complete analysis).

Paper URL: https://link.springer.com/article/10.1007/s10584-015-1580-2?wt_mc=internal.event.1.SEM.ArticleAuthorOnlineFirst

Data URL: https://figshare.com/articles/MICORDEA/1408501

Code URL: https://github.com/adamhsparks/MICCORDEA

Useful programming skills: R, Python, ArcGIS

17. Climate change may have limited effect on global risk of potato late blight.

Sparks, A. H., Forbes, G. A, Hijmans, R. J., & Garrett K. A. (2014). Climate change may have limited effect on global risk of potato late blight. Global Change Biology, doi:10.1111/gcb.12587.

submitted by Adam Sparks twitter github

Why should we attempt to reproduce this paper?

This is a two-for one. The repository contains code for companion papers, the model development and the model implementation and analysis. As the repository notes, some data are not freely available so I’ve made an effort to allow the paper to be replicated as best possible with what’s available.

Paper URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/gcb.12587

Data URL: https://figshare.com/articles/Supporting_files_for_Climate_change_may_have_limited_effect_on_global_risk_of_potato_late_blight/1066070

Code URL: https://github.com/adamhsparks/Global-Late-Blight-MetaModelling

Useful programming skills: R

18. Sea level regulated tetrapod diversity dynamics through the Jurassic/Cretaceous interval

Tennant, J. P., Mannion, P. D., & Upchurch, P. (2016). Sea level regulated tetrapod diversity dynamics through the Jurassic/Cretaceous interval. Nature Communications, 7, 12737.

submitted by Jon Tennant twitter github

Why should we attempt to reproduce this paper?

Because it’s a fun paper, involving dinosaurs! But one which I myself have also attempted to reproduce in the past, and struggled with. There are a few additional tweaks that might throw some people off too.

Paper URL: https://www.nature.com/articles/ncomms12737

Data URL: https://www.nature.com/articles/ncomms12737#supplementary-information

Code URL: https://www.nature.com/articles/ncomms12737#supplementary-information

Useful programming skills: R, Perl

19. Evaluation of App-Embedded Disease Scales for Aiding Visual Severity Estimation of Cercospora Leaf Spot of Table Beet

Del Ponte EM, Nelson SC, Pethybridge SJ (2019) Evaluation of App-Embedded Disease Scales for Aiding Visual Severity Estimation of Cercospora Leaf Spot of Table Beet. Plant disease 103:1347-1356. 10.1094/PDIS-10-18-1718-RE

submitted by Emerson M. Del Ponte twitter github

Why should we attempt to reproduce this paper?

There are data and code written in RMarkdown which allows to reproduce the entire analysis and plots shown of the paper. It also allows to generate HTML document, which is a nice interface that facilitates the reader to understand better why some procedures were adopted and how to run them.

Paper URL: https://apsjournals.apsnet.org/doi/10.1094/PDIS-10-18-1718-RE

Data URL: https://osf.io/ezxps/

Code URL: https://github.com/emdelponte/paper-estimate-app

Useful programming skills: R

20. Bayesian determination of the effect of a deep eutectic solvent on the structure of lipid monolayers

Phys. Chem. Chem. Phys., 2019,21, 6133-6141

submitted by Andrew McCluskey twitter github

Why should we attempt to reproduce this paper?

I believe this represents the only example of a reproducible paper from scattering data collected at Diamond Light Source (UK) and the Institute Laue-Langevin (France)

Paper URL: https://doi.org/10.1039/c9cp00203k

Data URL: https://doi.org/10.15125/BATH-00548

Code URL: https://doi.org/10.5281/zenodo.2577796

Useful programming skills: Python, make

21. A multiscale Bayesian inference approach to analyzing subdiffusion in particle trajectories

K. Hinsen and G.R. Kneller, J. Chem. Phys. 145, 151101 (2016)

submitted by Konrad Hinsen twitter github

Why should we attempt to reproduce this paper?

This is one of the very few papers in biomolecular simulation for which code and data are available and which should be reproducible. But it is also three years old, so it is an interesting test case for the longevity of reproducible research. The infrastructure software is available at http://www.activepapers.org/python-edition/ (with instructions for installation and use)

Paper URL: https://doi.org/10.1063/1.4965881

Data URL: https://doi.org/10.5281/zenodo.162171

Code URL: https://doi.org/10.5281/zenodo.162171

Useful programming skills: Python

22. Resolving the Measurement Uncertainty Paradox in Ecological Management

Memarzadeh, M., & Boettiger, C. (2019). Resolving the Measurement Uncertainty Paradox in Ecological Management. The American Naturalist, 193(5). https://doi.org/10.1086/702704

submitted by Carl Boettiger twitter github

Why should we attempt to reproduce this paper?

This will probably be a non-trivial example to reproduce, owing to: (1) long-running code, (2) dependency on external data sources, (3) possibly challenging software dependencies – both trivial ones (e.g. setting up custom fonts and plot themes) and critical ones (requires an external R package wrapping a C++ algorithm, not available on CRAN and can sometimes have interesting compiler issues, like when Apple decided to break the clang compiler in 10.0). Ideally one could just run the R code given in the appendix on your local R session, but that may take a bit of effort. We’ve tried to take steps to address those issues by providing caches of slow-running parts, providing a docker container, and providing sufficient annotations, but who knows!

Paper URL: https://doi.org/10.1086/702704

Data URL: NA

Code URL: https://github.com/boettiger-lab/pomdp-intro

Useful programming skills: R

23. Comparisons of Citizen Science Data-Gathering Approaches to Evaluate Urban Butterfly Diversity

Prudic KL, Oliver JC, Brown BV, Long EC. 2018. Comparisons of Citizen Science Data-Gathering Approaches to Evaluate Urban Butterfly Diversity. Insects. 9(4):E186. doi: 10.3390/insects9040186

submitted by Kathleen Prudic twitter github

Why should we attempt to reproduce this paper?

This is a fairly digestible paper with statistical analyses and data visualization that rely heavily on open data from citizen science projects.

Paper URL: https://doi.org/10.3390/insects9040186

Data URL: https://doi.org/10.5281/zenodo.1436741

Code URL: https://doi.org/10.5281/zenodo.1436741

Useful programming skills: R

24. Bivariate spatial point patterns in the retina: a reproducible review.

Eglen SJ (2016) Bivariate spatial point patterns in the retina: a reproducible review. Journal de la SociĂ©tĂ© Française de Statistique 157:33–48.

submitted by Stephen Eglen twitter github

Why should we attempt to reproduce this paper?

Tell me what I should improve!

Paper URL: https://github.com/sje30/eglen2015

Data URL: NA

Code URL: NA

Useful programming skills: R

25. A data repository and analysis framework for spontaneous neural activity recordings in developing retina

https://doi.org/10.1186/2047-217X-3-3

submitted by Stephen Eglen twitter github

Why should we attempt to reproduce this paper?

Tell me what I can improve on; maybe think of other visualisations for data?

Paper URL: https://doi.org/10.1186/2047-217X-3-3

Data URL: NA

Code URL: http://www.damtp.cam.ac.uk/user/eglen/waverepo/

Useful programming skills: R

26. Population structure and phenotypic variation of Sclerotinia sclerotiorum from dry bean (Phaseolus vulgaris) in the United States

Kamvar ZN, Amaradasa BS, Jhala R, McCoy S, Steadman JR, Everhart SE. 2017. Population structure and phenotypic variation of Sclerotinia sclerotiorum from dry bean (Phaseolus vulgaris) in the United States. PeerJ 5:e4152 https://doi.org/10.7717/peerj.4152

submitted by Zhian N. Kamvar twitter github

Why should we attempt to reproduce this paper?

This paper is reproduced weekly in a docker container on continuous integration, but it is also set up to work via local installs as well. It would be interesting to see if it’s reproducible with a human operator who knows nothing of the project or toolchain.

Paper URL: https://peerj.com/articles/4152/

Data URL: https://osf.io/k8wtm

Code URL: https://github.com/everhartlab/sclerotinia-366

Useful programming skills: R, Make and knowledge of Docker containers

27. PREPRINT: Using digital epidemiology methods to monitor influenza-like illness in the Netherlands in real-time: the 2017-2018 season

Schneider P, Van Gool C, Spreeuwenberg P, Hooiveld M, Donker GA, Barnett DJ, Paget J. Using digital epidemiology methods to monitor influenza-like illness in the Netherlands in real-time: the 2017-2018 season. BioRxiv. 2018 Jan 1:440867.

submitted by Paul Schneider twitter github

Why should we attempt to reproduce this paper?

This preprint is an attempt to reproduce Google Flu Trend in the Netherlands.

The whole paper + code is meant to be easily reproducible and transferable to other countries and/or areas. If you are familiar with time series data, lasso regression and cross validation, the analysis should be straight forward.

If anyone is interested, I could also provide influenza data for other European countries.

Paper URL: https://www.biorxiv.org/content/10.1101/440867v1.full

Data URL: https://zenodo.org/record/1459862#.XQbNIG8vNPM

Code URL: https://zenodo.org/record/1459862#.XQbNIG8vNPM

Useful programming skills: R

28. Growth Dynamics of Independent Gametophytes of Pleurosoriopsis makinoi ( Polypodiaceae)

Bulletin of the National Museum of Nature and Science, Series B (Botany) 45: 77–86

submitted by Joel Nitta twitter github

Why should we attempt to reproduce this paper?

It uses the drake R package that should make reproducibility of R projects much easier (just run make.R and you’re done). However, it does depend on very specific package versions, which are provided by the accompanying docker image.

Paper URL: https://www.joelnitta.com/publication/2019-03-27_pleurosoriopsis/

Data URL: https://github.com/joelnitta/pleurosoriopsis

Code URL: https://github.com/joelnitta/pleurosoriopsis

Useful programming skills: R

29. Open Trade Statistics

@misc{open_trade_statistics_2019, title = {OTS BETA DASHBOARD}, url = {https://shiny.tradestatistics.io/}, author = {{Open Trade Statistics}}, publisher = {Open Trade Statistics}, year = {2019}, month = {Apr}, note = {Accessed: June 22, 2019} }

submitted by Mauricio Vargas twitter github

Why should we attempt to reproduce this paper?

The focus of the project is reproducibility. Here we show the differences to access data compared to similar initiatives: https://ropensci.org/blog/2019/05/09/tradestatistics/. Also, similar projects have obscure parts, while our exposes the code from raw data downloading to dashboard creation.

Paper URL: https://shiny.tradestatistics.io

Data URL: https://api.tradestatistics.io

Code URL: https://github.com/tradestatistics

Useful programming skills: R, Shiny

30. model4you: An R Package for Personalised Treatment Effect Estimation

Seibold, H., Zeileis, A. and Hothorn, T., 2019. model4you: An R Package for Personalised Treatment Effect Estimation. Journal of Open Research Software, 7(1), p.17. DOI: http://doi.org/10.5334/jors.219

submitted by Heidi Seibold twitter github

Why should we attempt to reproduce this paper?

I guess it could be a cool learning experience. The paper is written with knitr, uses a seed, is part of the R package it describes, was openly written using version control (SVN, R-Forge) and is available in an open access journal (@up_jors).

Paper URL: http://doi.org/10.5334/jors.219

Data URL: NA

Code URL: https://r-forge.r-project.org/scm/viewvc.php/pkg/model4you/inst/JORS/?root=partykit

Useful programming skills: R, knitr (LaTeX), version control (SVN)

31. Comparing theory-driven and data-driven attractiveness models using images of real women’s faces

Holzleitner et al. (In press). Comparing theory-driven and data-driven attractiveness models using images of real women’s faces, JEP:HPP.

submitted by Ben Jones

Why should we attempt to reproduce this paper?

Complex analyses over multiple variables. In press, so we can still fix errors ahead of publication!!

Paper URL: https://psyarxiv.com/vhc5k

Data URL: https://osf.io/jurcq/

Code URL: https://osf.io/jurcq/

Useful programming skills: R


Program 📅

10:00 Coffee and Tea ☕

10:30 Welcome

10:40 “Reproducibility: The value in the practice” by Dr. Anna Krystalli

11:30 Forming groups and start hackathon đŸ‘šđŸ»â€đŸ’» đŸ‘šđŸŒâ€đŸ’»

12:30 Lunch buffet đŸ„— đŸ„™

14:30 “A vision for Open Science beyond the reproducibility crisis” by Dr. John Boy

15:00 Continue hacking đŸ‘šđŸœâ€đŸ’»đŸ‘©đŸŒâ€đŸ’»

16:30 Drinks and bites đŸ»


Feedback form

Please fill this feedback form after you reproduced the paper that you picked!

We want to emphasize that we appreciate all authors who have nominated their papers for the hackathon, and so we expect that any feedback and criticism on authors’ work is expressed in a constructive, friendly and respectful manner.

Find us:


https://twitter.com/ReproHackNL