Jupyter Notebook

Follow these three steps below to connect to a remote host jupyter notebook in your browser:

1. Submit a job for running Jupyter

Modify the lines with "#SBATCH" for your own needs. For example, you may need to request GPU nodes, or larger memory nodes.

#!/bin/bash
#SBATCH --nodes 1
#SBATCH --partition=defq
#SBATCH --mem 64G
#SBATCH --time 1-0:00:00
#SBATCH --job-name jupyter-notebook
#SBATCH --output jupyter-notebook-%J.log

# get tunneling info
XDG_RUNTIME_DIR=""
port=$(shuf -i8000-9999 -n1)
node=$(hostname -s)
user=$(whoami)
cluster=$(hostname -f | awk -F"." '{print $2}')

# print tunneling instructions jupyter-log
echo -e "
MacOS or linux terminal command to create your ssh tunnel:
ssh -N -L ${port}:${node}:${port} ${user}@${cluster}


Use a Browser on your local machine to go to:
localhost:${port}  (prefix w/ https:// if using password)
"

# load modules or conda environments here
# e.g. farnam:
# module load Python/2.7.13-foss-2016b

# DON'T USE ADDRESS BELOW.
# DO USE TOKEN BELOW
jupyter-notebook --no-browser --port=${port} --ip=${node}

Once the job get allocated resources, you should see something like this in a file named jupyter-notebook-${jobid}.log

[I 15:10:46.168 NotebookApp] Serving notebooks from local directory: /PATH/TO/YOUR/FOLDER
[I 15:10:46.168 NotebookApp] Jupyter Notebook 6.4.0 is running at:
[I 15:10:46.168 NotebookApp] http://esplhpc-cp006:8030/?token=fb22cf8ab11ca19017fbb5b602d3374e1f90f4aae6a0f811
[I 15:10:46.168 NotebookApp]  or http://127.0.0.1:8030/?token=fb22cf8ab11ca19017fbb5b602d3374e1f90f4aae6a0f811

From here, your node is esplhpc-cp006, your port is 8030

2. Configure local SSH Tunnel

Build the SSH tunnel using the following command on your local computer's terminal - remember to change USERNAME to your own username:

ssh -N -L 8030:esplhpc-cp006:8030 [e-mail]

Note that you can also use different login nodes esplhpccompbio-lv0[123].csmc.edu.

You can also add the following lines to your .bash_aliases for faster access and saving a few keystrokes:

csmc-jupyter() { NODE=$1; PORT=$2;
echo "$NODE:$PORT"; 
ssh -N -L $PORT:$NODE:$PORT [e-mail]; }

Then, you can connect by simply csmc-jupyter esplhpc-cp006 8030 on your local terminal.

3. Connect to remote Jupyter

Upon successful building the SSH tunel, you can connect to the running Jupyter notebook by simplying copy and paste the URL we saw in the first place. In this example, this is http://127.0.0.1:8030/?token=fb22cf8ab11ca19017fbb5b602d3374e1f90f4aae6a0f811

TODO: how can we put this in VSCode?

Note that you can connect to multiple running Jupyter notebook instances, as long as your ports are different.