Python, R, Reticulate, and Fixing Unknown Crashes (Numpy)

Python, R, Reticulate, and Fixing Unknown Crashes (Numpy)
Photo by Matt Hudson / Unsplash

A short post this time. If you use {reticulate} (the excellent library to bridge together Python and R code) you will sometimes end up with a complete system crash (R session aborted) when loading specific Python functions using a reticulate environment.

Since the error gives you no insight as to what is actually happening, this is a hard one to debug. Nobody likes to meet face-to-face with a "Segmentation Fault".

In my case, I have ultimately found that this error occurs when I use a Python environment with Numpy installed. As soon as the environment is loaded with Numpy, this leads to this kind of crash.

This happens (on Linux) when I install Numpy this way in a virtualenv:

reticulate::virtualenv_install(envname = "python-env-for-analysis",packages = "numpy")

To make this error go away, this is what actually works - you need to let pip know that you don't want to fetch a binary for numpy, but actually recompile it from sources on your local machine.

reticulate::virtualenv_install(envname = "python-env-for-analysis",packages = "numpy", pip_options = "--no-binary='numpy'",ignore_installed = TRUE)

It will take a little longer to install, but it's worth it.

If you ever get some weird, unexpected crashes with reticulate, keep this is in mind (or in a bookmark). It might save you many, many precious hours.