Using Dedalus on the NYU Greene Cluster#
Dedalus is a flexible differential equations solver using spectral methods. It is MPI-parallelized and therefore can make efficient use of high performance computing resources like the NYU Greene Cluster. The cluster uses Singularity containers to manage packages and Slurm for job scheduling. It is tricky to construct and use a Singularity container for Dedalus v3 that interacts with these well. Luckily, the NYU HPC staff has figured out much of the details. This note describes how to use the Singularity, on single node, on multiple nodes, and in JupyterLab. At the end we will also biefly comment on how to build the Singularity so that you can build your customized version.
Note
As I have graduated, I have less attention to maintain this guide. As Dedalus as well as Greene evolve, things are bound to break. You can open an issue. The NYU HPC team is amazing at troubleshooting. If there are parts of the guide that are out of date, please consider contributing.
Warning
The version of Dedalus used here is 3.0.0. It is nontrivial to work with a newer version where Dedalus uses MPI-4. See Building the Singularity for some details about building and using a singularity of the newest version of Dedalus.
Table of content#
Acknowledgment#
The Singularity files in this note are made by Shenglong Wang on the NYU HPC team. We thank the NYU HPC team for their help in training and troubleshooting.