Start Up Sequence
Your set up is complete. Here is the inventory of the items you prepared to execute this method to connect and run Jupyter Notebook from HYAK:
Element | klone or local | Notes |
---|---|---|
hyak-container.sif | klone | container image; directly or with a symbolic link |
conda-overlay.img | klone | container overlay with conda and jupyter environment; in read-write or read-only mode |
launch-container.sh | klone | launches container and overlay; in read-write or read-only mode |
start-jupyter-server.sh | klone | starts jupyter |
jupyter-server.job | klone | sbatch script that launches the container-overlay and jupyter as a SLURM job |
~/.ssh/config | local | creates ssh klone-login short cut |
~/.ssh/klone-node-config | local | creates ssh klone-node short cut for ProxyJump |
set-hyak-node.sh | local | replaces Hostname in ~/.ssh/klone-node-config for convenience |
start-jupyter-forwarding.sh | local | starts port forwarding and provides web address for Jupyter Notebook |
important
Before we begin the Start Up Sequence, make sure that you have no jobs called klone-container
running on klone
.
If you do, you can cancel the job with scancel
and the JobID:
Now we begin the Start Up Sequence:
#
1. Login to kloneLog in using your klone-login
short cut.
#
2. Navigate to your working directoryChange directory to the directory you have been using during this tutorial, where you have your materials stored. For example:
#
3. Start the Jupyter SLURM jobSubmit the jupyter-server.job
script with sbatch
to start a job running the container-overlay and jupyter.
Use the watch
command with the squeue
command to monitor the job in real time. watch -n10
will issue the squeue
every 10 seconds. Use Ctrl
+ C
to exit the watch
command.
Once the job is running, or you see R
under the column ST
using squeue
, use Ctrl
+ C
to exit the watch
command.
In your working directory, you will have a file called jupyter-server-XXXXXX.out
where the X's are the jobID assigned to the job by SLURM. Use the command tail --follow
to print the contents of the jupyter-server-XXXXXX.out
file and wait for the message, "Info: Jupyter server is running, port & token in ~/.jupyter-port-and-token." Once you see this message, your Jupyter server session is ready for the next step. (Use Ctrl
+ C
to exit the tail
command)
#
4. Prepare your ProxyJump to the compute nodecaution
The next steps are performed on your local computer.
Above we used squeue
to monitor the job we started. The NODELIST
column of the squeue
output shows the node where the job is running. This is the Hostname for our ~/.ssh/klone-node-config
script.
Let's try to execute the set-hyak-node.sh
script and see if it will replace our Hostname with n3219
where the job is running (the compute node will likely be different for you; n3219
is just an example).
If it didn't work, please use a text editor to manually change Hostname to the node where your job is running.
#
5. Start Port Forwardingcaution
The next steps are performed on your local computer.
Execute the start-jupyter-forwarding.sh
script to get the web address to use Jupyter Notebook in the browser.
Copy and paste the web address into your browser, and you should be connected to your Jupyter server where you can work in a Jupyter Notebook.
When you're finished, you can use the kill command we generated to ensure your port forwarding is stopped: