Scientific Reproducibility

Scientific Reproducibility

How to make your computational work redoable, by you in six months or by anyone else, with environments, data layout and records. Video 7 of 13 in the series Research Computing on Explorer and AICR, built from Northeastern Research Computing's training material. Liam, in for Bear, narrates.

Chapters 0:00 Opening 0:12 The idea 0:36 Terms in this lecture 0:59 I · Three Words 1:19 II · Why It Matters 2:16 III · The Checklist 3:06 IV · Record Your Software 3:49 V · Conda Environments 5:35 VI · Data 6:17 VII · Documentation 6:41 VIII · Start Tomorrow 7:14 Recap 7:45 Your turn

Your turn. Paste this into Claude: Here is how my current research project is organised and what software it uses: [describe it in three sentences]. Audit it against the reproducibility checklist: version control, environment file, folder layout, README, relative paths, random seeds, data README and dictionary, backups. List what is missing in order of how much it would hurt me six months from now, and write the environment.yml and the README skeleton for me. Then check two things yourself: did it pin versions in the yml? could a stranger follow the README without asking you anything?

Sources "Scientific Reproducibility: Best Practices and Implementation", Northeastern Research Computing, presenter Serena Caplins; github.com/northeastern-rc-training/Intro-to-Scientific-Reproducibility-2026. Read 2026-10-06. Questions about the clusters: rchelp@northeastern.edu.

Made with the brutalist.art toolkit: Manim, Remotion and Kokoro, free and local. Narration is synthetic (Liam, in for Bear). This is an independent educational film, not an official Northeastern publication.

▶ The full series playlist — Brutalist https://www.youtube.com/playlist?list=PLG9H-C6rp5RU

What this video covers

• 0:00 — Opening
• 0:12 — The idea
• 0:36 — Terms in this lecture
• 0:59 — I · Three Words
• 1:19 — II · Why It Matters
• 2:16 — III · The Checklist
• 3:06 — IV · Record Your Software
• 3:49 — V · Conda Environments
• 5:35 — VI · Data
• 6:17 — VII · Documentation
• 6:41 — VIII · Start Tomorrow
• 7:14 — Recap
• 7:45 — Your turn

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

Opening

[0:00] Research Computing on Explorer and AICR, scientific reproducibility. How to make your computational work redoable by you in 6 months or by anyone else with environments, data layout, and records.

The idea

[0:12] Hello, this is Liam in for Bear. First, a correction. Reproducibility is not a favor you do for other scientists. The first person it helps is you 6 months from now when a reviewer asks for one more plot. This lecture is a toolkit for making your research redoable. The three words people confuse, why the field has a crisis, a checklist, how to record your software, Conda environments done properly, data organization, documentation. And what to do tomorrow morning.

Terms in this lecture

[0:36] Six terms, version control software that records every change to your files so you can go back. A module, a piece of software the cluster team installed loaded by name and version. Conda, a tool that builds isolated environments of packages. An environment, one named set of software kept apart from all the others. A readme, the plain text file that explains a folder. And a random seed, the number that makes a random process repeat exactly.

I · Three Words

[0:59] Three words people mix up, reproducibility. Someone else takes your same data and your same methods and gets your result. Replicability. A different team collects new data, uses similar methods, and finds a comparable result. Repeatability. You yourself rerun the same methods on the same data 6 months later and get the identical answer. Today is mostly about the first and the third. Why it matters. In 2016, a Nature

II · Why It Matters

[1:21] Survey asked scientists whether they had failed to reproduce an experiment. Most had in chemistry, roughly nine in 10 had failed to reproduce someone else's result. In biology, nearly eight in 10. In physics, medicine, and Earth science, around two in three. Other studies put numbers on it. In psychology, only 36 to 47 of 100 studies replicated. In biomedicine, 11% of 53 landmark cancer studies. In economics, only 49% of papers even had enough data and code to try. It bites in the real world. Clinical decisions built on results no one can reproduce. Policy built on models no one can rerun, and it bites you first. In every project, something has to be redone, an analysis, a plot. The more reproducible your workflow, the faster that is, and the faster you publish. The usual culprits are five: methods not documented, data and code not shared, software versions not recorded, results that depend on one machine, and absolute file paths baked into code.

III · The Checklist

[2:16] The checklist in three phases: at setup, a Git repository, a consistent folder layout, a documented environment, a readme. During the work, commented code, relative paths, random seed set, every data transformation written down, regular commits. At the end, have someone else run it, ship the environment file and the instructions, share the data and the code, and write down the limitations. And one principle to carry, FAIR, set out in Scientific Data in 2016. Findable, persistent identifiers and metadata, accessible, stored in open repositories, interoperable, standard formats, reusable, clear licensing and documentation. Version control gets its own lecture, but the idea in one breath, every change is recorded, you can revert to any earlier version, collaborate without overriding each other, keep the whole history, and experiment on a branch without breaking the code that works.

IV · Record Your Software

[3:06] Record your software. If you load a cluster module, write down the version and when you ran it in the job script or a readme. A job script like this one is already a record. Julia 1.10.4, BW 0.7.18, BAMtools 2.5.2. Change it and copy the file first, or let Git keep the versions. The team's own module install scripts are public on GitHub, so you can see exactly how a module was built. And don't load modules in your .bashrc. That hidden file runs in every shell and every program you open on the cluster. Aliases and colors belong there. Modules don't. They break the self-contained job script, so later you can't tell whether a program came from the script or the shell, and they cause dependency conflicts. Keep software in scripts and

V · Conda Environments

[3:49] Readme. Conda builds isolated environments. It works on every platform, handles Python and non-Python packages, keeps each project's dependencies apart, and resolves conflicts. On the cluster, get a compute node, load a miniconda module, create an environment with a Python version, activate it with source activate, install, and deactivate when done. Seven habits: pin the versions that matter. Unpinned, Conda takes the latest. Prefer the conda-forge channel, the best maintained. Channel order matters. Conda searches top to bottom. Install Conda packages first, then pip ones. Export the environment with from history for a cleaner file. Install everything in one session because Conda resolves conflicts across the whole set at once, and an environment grown over months is the one that won't rebuild. And keep environments small, one per analysis, not one to rule them all. Picture the two ways to build one. Added to over months, package by package, the environment collects conflicts, and when you

Try to recreate it, it won't resolve. Built in one session from one file, it resolves once and rebuilds anywhere. Managing them: list your environments, create one at a specific path in your project space, remove one, clone one as a backup, update one from its YML file, pruning what the file no longer lists. If solving is slow, install Mamba and use it in place of Conda. And always use source activate. Conda activate needs Conda in it, which rewrites your .bashrc and fights with open on demand. The complete workflow starts from a file. One environment.yml names the environment, the channels in order, the pinned Python and packages, and the pip extras last. Make the project folder, start git, create the environment from the file, activate it, work, export the final environment, and commit both files. In a job script, the same two lines, module load and source activate, make the run reproducible. Data: three

VI · Data

[5:35] Folders, raw, which is original and read-only, processed, cleaned, and transformed, and metadata, dictionaries and sample ID, each with its own readme. Make raw read-only with chmod 555 with your professor's agreement since only the owner can change permissions and name files to sort themselves. ISO dates, version numbers, lowercase underscores or hyphens, never spaces. Every data folder needs a readme, where the data came from, what each variable means and its range, what processing was done and what's known to be wrong with it. A data dictionary makes the variables a table, name, type, description, units, range and the missing code. And back it all up by the 3-2-1 rule, three copies on two different media, one of them offsite. The project readme, what it is

VII · Documentation

[6:19] In two sentences, how to install, how to run, where the data is and the reproduction steps as a numbered list someone can follow without you. Then citation, license, contact. In code, give functions docstrings, what it does, parameters, what it returns and comment the why, not the what. Adjust for one-based indexing in the original data set tells a reader something. Add one to X does not.

VIII · Start Tomorrow

[6:42] Five things to keep in the exercise. Use version control from day one. Document your environment with conda or pip. Organize files systematically, data, source, results. Write the documentation, readme, comments, dictionaries and test for reproducibility by having someone else run your code. The exercise, write an environment.yml with Python 3.10, numpy, pandas and matplotlib. Build it, activate it, install a couple more packages. Export the final file. Start the next project this way. Document as you go and share when you can. Perfect reproducibility is hard, any improvement counts.

Recap

[7:14] Let's recap with Claude. Reproducible means same data, same methods and someone else gets your result. Most scientists have failed to reproduce a result and the first person you help is yourself. At setup, get a layout, an environment file, a readme. During, relative paths, seeds, commits, job scripts, record module versions and modules never go in .bashrc. Conda pin versions, prefer conda-forge, install in one session. Keep environments small, export them and use source activate. Data, raw, read-only, readme, dictionaries, 321 backups, and comment the why. Your turn. Paste this

Your turn

[7:46] Into Claude. Here's how my current research project is organized and what software it uses. Describe it in three sentences. Audit it against the reproducibility checklist, version control, environment file, folder layout, read me, relative paths, random seeds, data read me, and dictionary backups. List what is missing in order of how much it would hurt me 6 months from now, and write the environment. Immol and the read me skeleton for me, then check two things yourself. Did it pin versions in the YML, and could a stranger follow the read me steps without asking you anything? Scientific reproducibility at Nick Bear Brown.