Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TRT-558 - Update service to use earthdata-varinfo==3.0.0. #7

Merged
merged 3 commits into from
Sep 23, 2024
Merged

TRT-558 - Update service to use earthdata-varinfo==3.0.0. #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e98fc3d
TRT-558 - Update service to use earthdata-varinfo==3.0.0.
owenlittlejohns Sep 23, 2024
7988750
TRT-558 - Update local Docker image names to use harmony-regridding-s…
owenlittlejohns Sep 23, 2024
c1ca806
TRT-558 - Simplify service name in base service exception.
owenlittlejohns Sep 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@ The Harmony Regridding Service follows semantic versioning. All notable changes
to this project will be documented in this file. The format is based on [Keep a
Changelog](http://keepachangelog.com/en/1.0.0/).

## [v1.0.2] - 2024-09-23

### Changed
This version of the Harmony Regridding Service updates to use
`earthdata-varinfo==3.0.0`, which primarily requires updates to the
configuration file the Harmony Regridding Service uses to specify metadata
overrides and excluded science variables to `earthdata-varinfo`.

This update also ensures that the Docker images used for local testing are
named `ghcr.io/nasa/harmony-regridding-service` and
`ghcr.io/nasa/harmony-regridding-service-test`, to comply with the released
image name for the service, and the name that Harmony-in-a-Box is looking for.

## [v1.0.1] - 2024-06-20

### Changed
Expand All @@ -22,6 +35,6 @@ include updated documentation and files outlined by the
For more information on internal releases prior to NASA open-source approval,
see legacy-CHANGELOG.md.

[unreleased]: https://github.com/nasa/harmony-regridding-service/compare/1.0.1..HEAD
[v1.0.1]: https://github.com/nasa/harmony-regridding-service/compare/1.0.0..1.0.1
[v1.0.0]: https://github.com/nasa/harmony-regridding-service/compare/0.0.4-legacy..1.0.0
[v1.0.2]: https://github.com/nasa/harmony-regridding-service/releases/tag/1.0.2
[v1.0.1]: https://github.com/nasa/harmony-regridding-service/releases/tag/1.0.1
[v1.0.0]: https://github.com/nasa/harmony-regridding-service/releases/tag/1.0.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flamingbear - I think this is the transition you made in one of the recent change logs. Let me know if this is the preferred URL format you like, or if I should update to something else.

3 changes: 2 additions & 1 deletion bin/build-image
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@
# 2022-01-03: Updated Dockerfile path and added comments for tags.
# 2023-01-26: Updated for the Harmony Regridding service.
# 2024-04-12: Updated image name to: ghcr.io/nasa/harmony-regridder.
# 2024-09-23: Updated image name to: ghcr.io/nasa/harmony-regridding-service.
#
###############################################################################

image="ghcr.io/nasa/harmony-regridder"
image="ghcr.io/nasa/harmony-regridding-service"

# Retrieve the tag from the script arguments, or default to "latest". Bamboo
# will pass the contents of `docker/service_version.txt` as this argument,
Expand Down
9 changes: 5 additions & 4 deletions bin/build-test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,19 @@
# Build a Docker container to run the test suite for the Harmony Regridding
# service.
#
# To build the ghcr.io/nasa/harmony-regridder-test image, the
# ghcr.io/nasa/harmony-regridder image must also be present locally, as it is
# used as the base image in `docker/tests.Dockerfile`.
# To build the ghcr.io/nasa/harmony-regridding-service-test image, the
# ghcr.io/nasa/harmony-regridding-service image must also be present locally,
# as it is used as the base image in `docker/tests.Dockerfile`.
#
# 2020-05-07: Adapted from SwotRepr.
# 2022-01-03: Updated Dockerfile path.
# 2023-01-26: Updated for the Harmony Regridding service.
# 2024-04-12: Updated image name to: ghcr.io/nasa/harmony-regridder-test.
# 2024-09-23: Updated image name to: ghcr.io/nasa/harmony-regridding-service-test.
#
###############################################################################

image="ghcr.io/nasa/harmony-regridder-test"
image="ghcr.io/nasa/harmony-regridding-service-test"
tag=${1:-latest}


Expand Down
10 changes: 6 additions & 4 deletions bin/clean-images
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
#!/bin/bash
###############################################################################
#
# Look for all versions of images containing the string "ghcr.io/nasa/harmony-regridder"
# and remove them. This is primarily for clean-up after development.
# Look for all versions of images containing the string
# "ghcr.io/nasa/harmony-regridding-service" and remove them. This is primarily
# for clean-up after development.
#
# 2020-05-07: Adapted from SwotRepr.
# 2023-01-26: Updated for the Harmony Regridding Service.
# 2024-04-12: Update to account for new open-source Docker image names.
# 2024-04-12: Updated to account for new open-source Docker image names.
# 2024-09-23: Updated Docker image name to "ghcr.io/nasa/harmony-regridding-service".
#
###############################################################################
remove_image_by_name() {
Expand All @@ -18,7 +20,7 @@ remove_image_by_name() {
fi
}

image_base_name="ghcr.io/nasa/harmony-regridder"
image_base_name="ghcr.io/nasa/harmony-regridding-service"

# First remove test images:
remove_image_by_name "${image_base_name}-test"
Expand Down
7 changes: 4 additions & 3 deletions bin/run-test
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
#!/bin/bash
###############################################################################
#
# Execute the ghcr.io/nasa/harmony-regridder-test Docker image as a container,
# either locally or within a GitHub Action.
# Execute the ghcr.io/nasa/harmony-regridding-service-test Docker image as a
# container, either locally or within a GitHub Action.
#
# 2020-05-07: Adapted from SwotRepr.
# 2021-01-05: Updated for JUnit output.
# 2023-01-26: Updated for the Harmony Regridding service.
# 2024-04-12: Updated to use new open-source Docker image names.
# 2024-09-23: Updated to use renamed test Docker image.
#
###############################################################################

Expand All @@ -27,4 +28,4 @@ mkdir -p coverage
docker run --rm \
-v $(pwd)/test-reports:/home/tests/reports \
-v $(pwd)/coverage:/home/tests/coverage \
ghcr.io/nasa/harmony-regridder-test "$@"
ghcr.io/nasa/harmony-regridding-service-test "$@"
4 changes: 2 additions & 2 deletions docker/service.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
###############################################################################
#
# Service image for ghcr.io/nasa/harmony-regridder, a Harmony backend service
# that transforms Level 3 or Level 4 data to another grid.
# Service image for ghcr.io/nasa/harmony-regridding-service, a Harmony backend
# service that transforms Level 3 or Level 4 data to another grid.
#
# This image installs dependencies via Pip. The service code is then copied
# into the Docker image.
Expand Down
11 changes: 6 additions & 5 deletions docker/tests.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
###############################################################################
#
# Test image for the Harmony Regridding service. This test image uses the main
# service image, ghcr.io/nasa/harmony-regridder, as a base layer for the tests.
# This ensures that the contents of the service image are tested, preventing
# discrepancies between the service and test environments.
# service image, ghcr.io/nasa/harmony-regridding-service, as a base layer for
# the tests. This ensures that the contents of the service image are tested,
# preventing discrepancies between the service and test environments.
#
# 2023-01-26: Added to repository.
# 2024-04-12: Update base image to ghcr.io/nasa/harmony-regridder.
# 2024-04-12: Updated base image to ghcr.io/nasa/harmony-regridder.
# 2024-09-23: Updated base image name to ghcr.io/nasa/harmony-regridding-service.
#
###############################################################################
FROM ghcr.io/nasa/harmony-regridder
FROM ghcr.io/nasa/harmony-regridding-service

ENV PYTHONDONTWRITEBYTECODE=1

Expand Down
4 changes: 2 additions & 2 deletions docs/Harmony_Regridding_Service_User_Documentation.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"\n",
"The Harmony Regridding Services is a [Harmony](https://harmony.earthdata.nasa.gov) backend service that transforms the data within a specified input granule to a single geographic grid. The initial version of this service is only compatible with geographic source and target grids. The transformed output is returned to the user in the form of a standard Harmony STAC object, which contains links to the output granule in an AWS S3 bucket.\n",
"\n",
"The Harmony Regridding Service leverages the `sds-varinfo` Python package (see: [here](https://git.earthdata.nasa.gov/projects/SITC/repos/sds-varinfo/browse)) to map variable dependencies and group variables by horizontal spatial dimensions. `pyresample` (see [here](https://pyresample.readthedocs.io/en/latest/)) is used to perform the resampling in a horizontal spatial plane.\n",
"The Harmony Regridding Service leverages the `earthdata-varinfo` Python package (see: [here](https://github.com/nasa/earthdata-varinfo)) to map variable dependencies and group variables by horizontal spatial dimensions. `pyresample` (see [here](https://pyresample.readthedocs.io/en/latest/)) is used to perform the resampling in a horizontal spatial plane.\n",
"\n",
"\n",
"## Capabilities\n",
Expand Down Expand Up @@ -292,7 +292,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
"version": "3.11.10"
}
},
"nbformat": 4,
Expand Down
58 changes: 22 additions & 36 deletions harmony_regridding_service/config/HRS_varinfo_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"Identification": "trajectorysubsetter_varinfo_config",
"Version": 2,
"Collection_ShortName_Path": [
"Identification": "harmony_regridding_service_varinfo_config",
"Version": 3,
"CollectionShortNamePath": [
"/HDF5_GLOBAL/short_name",
"/NC_GLOBAL/short_name",
"/Metadata/DatasetIdentification/shortName",
Expand All @@ -16,58 +16,44 @@
"Mission": {
"ATL\\d{2}": "ICESat2"
},
"Excluded_Science_Variables": [
"ExcludedScienceVariables": [
{
"Applicability": {
"Mission": "ICESat2"
},
"Variable_Pattern": [
"VariablePattern": [
"/quality_assessment/.*",
"/orbit_info/.*",
"/atlas_impulse_response/.*"
]
}
],
"ProductEpochs": [
"MetadataOverrides": [
{
"Applicability": {
"Mission": "ICESat2"
"Mission": "ICESat2",
"ShortNamePath": "ATL14",
"VariablePattern": "/tile_stats/x"
},
"Epoch": "2005-01-01T00:00:00.000000"
}
],
"CF_Overrides": [
"Attributes": [
{
"Name": "standard_name",
"Value": "projection_x_coordinate"
}
]
},
{
"Applicability": {
"Mission": "ICESat2",
"ShortNamePath": "ATL14"
"ShortNamePath": "ATL14",
"VariablePattern": "/tile_stats/y"
},
"Applicability_Group": [
"Attributes": [
{
"Applicability": {
"Variable_Pattern": "/tile_stats/x"
},
"Attributes": [
{
"Name": "standard_name",
"Value": "projection_x_coordinate"
}
]
},
{
"Applicability": {
"ShortNamePath": "ATL14",
"Variable_Pattern": "/tile_stats/y"
},
"Attributes": [
{
"Name": "standard_name",
"Value": "projection_y_coordinate"
}
]
"Name": "standard_name",
"Value": "projection_y_coordinate"
}
]
}
],
"CF_Supplements": []
]
}
2 changes: 1 addition & 1 deletion harmony_regridding_service/exceptions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ class RegridderException(HarmonyException):
"""Base service exception."""

def __init__(self, message=None):
super().__init__(message, 'sds/harmony-regridder')
super().__init__(message, 'harmony-regridding-service')


class InvalidTargetCRS(RegridderException):
Expand Down
2 changes: 1 addition & 1 deletion pip_requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Open source packages available from PyPI
earthdata-varinfo ~= 2.0.0
earthdata-varinfo ~= 3.0.0
harmony-service-lib ~= 1.0.25
numpy ~= 1.24.2
netCDF4 ~= 1.6.3
Expand Down
Loading