Skip to content

Commit

Permalink
general improvements made while reviewing code. (#31)
Browse files Browse the repository at this point in the history
  • Loading branch information
flamingbear authored Apr 27, 2023
1 parent cee4eb6 commit 5fc6aa1
Show file tree
Hide file tree
Showing 5 changed files with 28 additions and 60 deletions.
4 changes: 4 additions & 0 deletions CHANGE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[version 1.2.12] 2023-04-26
* Rename function `combin_transfer` -> `perform_transforms`
* Removes unused code module `capture_operation.py`
* Tidies Readme
[version 1.2.11] 2022-12-23
* Refactor `HarmonyAdapter.process_item` to move staging and STAC record
generation to separate class method.
Expand Down
25 changes: 12 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,16 @@

![](https://data-services-github-badges.s3.amazonaws.com/cov.svg?dummy=true)

A Harmony (https://harmony.earthdata.nasa.gov/) backend service that transforms
A [Harmony](https://harmony.earthdata.nasa.gov/) backend service that transforms
input images using GDAL.

HGA is published to [ghcr.io](https://github.com/nasa/harmony-gdal-adapter/pkgs/container/harmony-gdal-adapter)
GitHub's Container registry.
GitHub's Container Registry.

HGA is invoked by [harmony](https://github.com/nasa/harmony)
when the harmony server is configured, via Harmony's [service.yml](https://github.com/nasa/harmony/blob/main/config/services.yml)
or by UMM-S/C associations in CMR, to handle an incoming request for the
collection. You can see examples of requests that harmony dispatches to the
harmony-gdal-adapter by examining the [regression test notebook for hga](https://github.com/nasa/harmony-regression-tests/blob/main/test/hga/HGA_regression.ipynb).
when the harmony server is configured by UMM-Service UMM-Collection associations in CMR,
to handle an incoming request for the collection. You can see examples of requests that harmony dispatches to the
harmony-gdal-adapter by examining the [regression test notebook for hga](https://github.com/nasa/harmony-regression-tests/blob/main/test/hga/HGA_Regression.ipynb).


## Test with Docker
Expand All @@ -33,23 +32,24 @@ Creates the `nasa/harmony-gdal-adapter-test` test image.
```bash
bin/run-test
```
The `run-test` script mounts `test-reports` and `coverage` directories and run
the test script `tests/run_tests.sh` inside of a Docker test container.

The `run-test` script mounts `test-reports` and `coverage` directories and runs
the test script `tests/run_tests.sh` inside of the Docker test container.


## Test Locally

### Create isolated environment

```bash
conda create --name hga python=3.8 --channel conda-forge
conda create --name hga python=3.10 --channel conda-forge
conda activate hga
```

### Install requirements

```bash
conda install gdal==3.4.3
conda install gdal==3.6.4
pip install -r requirements.txt -r requirements_dev.txt
```

Expand Down Expand Up @@ -144,9 +144,8 @@ within this repository. Other developers will need to follow the
[fork-and-pull model](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/getting-started/about-collaborative-development-models#fork-and-pull-model).

In addition to any DAAC stakeholders, please add members of the EED Data
Services team (currently: David Auty, Ken Cockerill, Owen Littlejohns and Matt
Savoie) as PR reviewers. One of these developers must approve the PR before
it is merged.
Services team (currently: David Auty, Owen Littlejohns and Matt Savoie) as PR
reviewers. One of these developers must approve the PR before it is merged.

NASA GitHub contributers will have the required GitHub permissions to merge
their PRs after the peer review is complete. Please consider using the
Expand Down
42 changes: 0 additions & 42 deletions gdal_subsetter/capture_operation.py

This file was deleted.

15 changes: 11 additions & 4 deletions gdal_subsetter/transform.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ def process_geotiff(self, source: HarmonySource, basename: str,
filename = input_filename
layer_id = basename + '__all'
band = None
layer_id, filename, output_dir = self.combin_transfer(
layer_id, filename, output_dir = self.perform_transforms(
layer_id, filename, output_dir, band)

filelist.append(filename)
Expand All @@ -232,7 +232,7 @@ def process_geotiff(self, source: HarmonySource, basename: str,
if band:
filename = input_filename
layer_id = basename + '__' + variable.name
layer_id, filename, output_dir = self.combin_transfer(
layer_id, filename, output_dir = self.perform_transforms(
layer_id, filename, output_dir, band)
layernames.append(layer_id)
filelist.append(filename)
Expand Down Expand Up @@ -263,7 +263,7 @@ def process_netcdf(self, source: HarmonySource, basename:str,
# convert a subdataset in the nc file into the GeoTIFF file
filename = self.nc2tiff(layer_id, filename, output_dir)
if filename:
layer_id, filename, output_dir = self.combin_transfer(
layer_id, filename, output_dir = self.perform_transforms(
layer_id, filename, output_dir, band
)

Expand Down Expand Up @@ -378,6 +378,7 @@ def varsubset(self, srcfile, dstfile, band=None):
return dstfile

def subset(self, layerid, srcfile, dstdir, band=None):
"""Subset layer to region defined in message.subset."""
normalized_layerid = layerid.replace('/', '_')
subset = self.message.subset
if subset.bbox is None and subset.shape is None:
Expand Down Expand Up @@ -477,6 +478,11 @@ def get_shapefile(self, subsetshape, dstdir):
return outfile

def resize(self, layerid, srcfile, dstdir):
"""Resizes the input layer
Uses a call to gdal_translate using information in message.format.
"""
interpolation = self.message.format.process('interpolation')
if interpolation in resampling_methods:
resample_method = interpolation
Expand Down Expand Up @@ -804,7 +810,8 @@ def get_variables(self, filename: str):

return result

def combin_transfer(self, layer_id, filename, output_dir, band):
def perform_transforms(self, layer_id, filename, output_dir, band):
"""Push layer through the series of transforms. """
filename = self.subset(layer_id, filename, output_dir, band)
filename = self.reproject(layer_id, filename, output_dir)
filename = self.resize(layer_id, filename, output_dir)
Expand Down
2 changes: 1 addition & 1 deletion version.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.2.11
1.2.12

0 comments on commit 5fc6aa1

Please sign in to comment.