2. Take 2 Set up JupyterHub environment

Notes

Setting up the environment means installing the various python packages (and Python version). In my case, I don't need multiple kernels (different Python versions and different package versions). I have a base environment that all the users will use.

See Take 1 for my first, partially successful, attempt

Note, on a t2.micro the environment steps are horribly slow and tend to crash the instance. If you can get TLJH running on a bigger instance, you'll save yourself some headaches. Update: I did get TLJH running on a t2.small instance by first installing on t2.micro and then after TLJH was installed, I upgraded the instance to t2.small. So try this. 1. Follow TLJH installation instructions and select t2.micro. 2. Test that you can get into your new hub. 3. Stop the instance and change to something bigger (t2.small or t2.medium). 4. Test and if ok, try the environment installation code.

Setting up my JupyterHub Environment

Note, I first used conda to try to install the environment, but that was using too much RAM. I only have 1G RAM on t2.micro.

I am going to have one environment, the base, rather than multiple. All the users will be working in that base environment.

1. From the JupyterHub console, create a new text file at the base level. Rename it environment.yml. Then copy the NASA Openscapes file into that https://github.com/NASA-Openscapes/corn/blob/main/ci/environment.yml Then change the first line of that file to name: base.

2. In a bigger instance (not t2.micro), it should work to just run this code to download all the packages with the same versions as in the NASA workshop:

   sudo -E mamba env update -f environment.yml
   

   But this crashes the t2.micro instance. I am pretty sure this will work on a t2.small. With a t2.micro my workaround was to install the environment in smaller pieces.

3. Workaround for environment creation crashing on t2.micro. I made env1.yml with the first 5 packages in environment.yml and ran

   sudo -E mamba env update -f env1.yml
   

   Then I added gdal to env1.yml and reran the command above to update the environment. I kept doing that, adding 1-2 packages at a time, until I had added all the packages in environment.yml. Yeah, doing this on a t2.micro is painful. It took forever but adding one package at a time meant I didn't crash the t2.micro instance. Note, make sure you have enough storage; the default 8Gig is not enough to install all the packages used in the NASA workshop.

   There were a few packages that crashed even if I tried installing so I commented these out of the environment file. I get a Verifying transaction: \ Killed error at the end.

   • xarray-spatial
   • geoviews

   However after I installed all the other packages except these, I was able to go back and uncomment geoviews and it worked. xarray-spatial still gave me that error. And after I upgraded my instance to a t2.small, I was able to install xarray-spatial.

Add harmony-py

pip install -U harmony-py

Manually installing packages

Note, why not do all the installations one by one? I don't know. Because this is a single-user installation, we have to use sudo. This would not be needed if we weren't using TLHJ.

Install one package

sudo -E mamba install -c conda-forge <packagename>

Install with pip

sudo -E pip install <packagename>

Pip complained about 'sudo' but I wanted packages available at the root level. sudo pip install <pkg> (without the -E) seemed to crash the instance.