Keeping Analysis Work Reproducible

When I look back at some of my older analysis projects, I notice a pattern. The code may still be there, the data may still be somewhere on disk, but the exact way I produced the results is often unclear. A small change to a script, a new version of a library (looking at you Awkward), or moving files to a different location can be enough to make it impossible to recreate the output exactly as before. This is not only frustrating, it can also delay work when you need to validate results or respond to a question. Over time, I learned that reproducibility is not something you add at the end, but something you build into the workflow from the start.

The first step is to keep both code and data under version control where possible. For code this is simple with Git or similar tools. Every commit records what changed and when. For data, it can be more complex because datasets can be large. Even if you cannot store the full data in the repository, you can store metadata about it. This includes file names, sizes, checksums, and download locations. A checksum is especially useful because it gives you a fingerprint of the file so you can be sure it has not changed over time. If the dataset is generated locally, record the exact steps or scripts that created it so you can run them again if needed.

Clear documentation is the second piece of the puzzle. A short README file in each project folder is more valuable than it sounds. It should include a description of the purpose of the project, the expected environment, and the commands needed to run the analysis. This does not have to be formal, but it should be specific. Instead of “run the script”, write the exact command with all the arguments. If there are multiple stages, list them in order. This is as much for you as it is for others. Six months from now, you will not remember every little detail.

Managing the environment is another critical step. Different machines may have different versions of the same library, which can change results in subtle ways. For Python projects, a requirements.txt file with exact versions ensures consistency. For C++ projects, a CMake preset can lock down compiler flags and library paths. If you use containers like Docker or Podman, you can freeze the environment completely, making it portable between systems. Even if you do not go that far, just having a script to install the right versions of dependencies saves time and removes guesswork.

Intermediate results are worth keeping when they take a long time to compute. For example, if you have a step that takes hours to run, save its output in a well-named folder along with a note on how it was generated. This way, you can skip that step when making small changes later. The key is to make sure these files are traceable back to the exact code and inputs that produced them. If the code changes, regenerate them so you do not mix incompatible data.

Logging may seem minor, but it plays a huge role in reproducibility. Each time you run an analysis, the log should capture the date, the parameters used, and any warnings or errors. Store these logs alongside the results. If a result ever comes into question, you can go back and see exactly what inputs were used and how the run behaved. This also helps when debugging issues that only appear under certain conditions.

Automation is the final habit that ties it all together. The more steps you automate, the less room there is for human error. Ideally, you should be able to start from raw data and end with the final plots or tables by running one script or makefile target. If there are manual steps that cannot be avoided, document them clearly and keep them consistent. Scripts that chain together all the steps also serve as living documentation of the workflow.

The following table can help with a summary of the process

Reproducibility is not just a box to check for formal reports or publications. It is a way to make your work easier to maintain, easier to share, and more resilient to time. A project that is reproducible is a project you can leave for months and return to without losing a day trying to remember what you did. It is also a gift to collaborators who need to understand or extend your work. In the long run, building reproducibility into your workflow saves far more time than it costs.