Skip to content

2. Take 2 Set up JupyterHub environment

Jump to bottom
Eli Holmes edited this page Dec 1, 2021 · 9 revisions

Notes

Setting up the environment means installing the various python packages (and Python version). In my case, I don't need multiple kernels (different Python versions and different package versions). I have a base environment that all the users will use.

See Take 1 for my first, partially successful, attempt

Note, on a t2.micro the environment steps are horribly slow and tend to crash the instance. If you can get TLJH running on a bigger instance, you'll save yourself some headaches.

Setting up my JupyterHub Environment

Note, I first used conda to try to install the environment, but that was using too much RAM. I only have 1G RAM on t2.micro.

I am going to have one environment, the base, rather than multiple. All the users will be working in that base environment.

  1. From the JupyterHub console, create a new text file at the base level. Rename it environment.yml. Then copy the NASA Openscapes file into that https://github.com/NASA-Openscapes/corn/blob/main/ci/environment.yml Then change the first line of that file to name: base.

  2. In a bigger instance (not t2.micro), it should work to just run this code to download all the packages with the same versions as in the NASA workshop:

    sudo -E mamba env update -f environment.yml

    But this crashes the t2.micro instance. Workaround was to install the environment in smaller pieces.

  3. Workaround for environment creation crashing. I made env1.yml with the first 5 packages in environment.yml and ran

    sudo -E mamba env update -f env1.yml

    Then I added gdal to env1.yml and reran the command above to update the environment. I kept doing that, adding 1-2 packages at a time, until I had added all the packages in environment.yml. Yeah, doing this on a t2.micro is painful. It took forever but adding one package at a time meant I didn't crash the t2.micro instance. Note, make sure you have enough storage; the default 8Gig is not enough to install all the packages used in the NASA workshop.

    There were a few packages that crashed even if I tried installing so I commented these out of the environment file. I get a Verifying transaction: \ Killed error at the end.

    • xarray-spatial
    • geoviews

    However after I installed all the other packages except these, I was able to go back and uncomment geoviews and it worked. xarray-spatial still gave me that error.

Manually installing packages

Note, why not do all the installations one by one? I don't know. Because this is a single-user installation, we have to use sudo. This would not be needed if we weren't using TLHJ.

Install one package

sudo -E mamba install -c conda-forge <packagename>

Install with pip

sudo -E pip install <packagename>

Pip complained about 'sudo' but I wanted packages available at the root level. sudo pip install <pkg> (without the -E) seemed to crash the instance.

Clone this wiki locally