EDSC-3881: Create portal for CASEI #1699

rushgeo · 2023-12-05T22:09:44Z

Overview

What is the feature?

Add a portal to EDSC for CASEI, the Catalog of Archived Suborbital Earth Science Investigations.

What is the Solution?

A new portal was created using a query for the 107 projects/campaigns currently available from CASEI.

Note that if new projects are added to CASEI, we will need to update this portal config.

We have an old unused portal called Suborbital that looks like it was intended to be use used for CASEI. I tested it, but its query returned fewer collections (669).

Testing

Click Browse Portals
Ensure there is an option for CASEI, then click it to enter the portal
There should be around 1,768 matching collections

Attachments

CASEI added to the list of portals:

After entering portal:

Checklist

I have added automated tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings

codecov · 2023-12-05T22:21:14Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6464b9d) 91.94% compared to head (fb5838c) 91.94%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1699   +/-   ##
=======================================
  Coverage   91.94%   91.94%           
=======================================
  Files         725      725           
  Lines       19353    19353           
  Branches     4561     4561           
=======================================
  Hits        17795    17795           
  Misses       1422     1422           
  Partials      136      136

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

macrouch · 2023-12-06T00:19:58Z

portals/casei/config.json

+  "parentConfig": "edsc",
+  "portalBrowser": true,
+  "query": {
+    "project": "ABoVE&project=SnowEx&project=Delta-X&project=S-MODE&project=IMPACTS&project=SASSIE&project=CPEX-CV&project=ACCLIP&project=CALIPSO-NVF&project=DCOTSS&project=ACTIVATE&project=MOOSE&project=CPEX-AW&project=TRACER-AQ&project=OMG&project=EXPORTS&project=CAMP2&project=FIREX-AQ&project=LISTOS&project=ACT-America&project=SCOAPE&project=Aeolus&project=ORACLES&project=HyspIRI&project=OWLETS&project=ATom&project=NAAMES&project=ICE-POP&project=SPURS&project=ACEPOL&project=EPOCH&project=ASCENDS&project=CPEX&project=LMOS&project=CORAL&project=CARAFE&project=GOES-R&project=POSIDON&project=KORUS-AQ&project=KORUS-OC&project=OLYMPEX&project=AfriSAR&project=RADEX&project=CARVE&project=VIRGAS&project=AirMOSS&project=COMEX&project=Polar&project=ATTREX&project=CalWater&project=HS3&project=ARISE&project=DISCOVER-AQ&project=SABOR&project=IPHEx&project=SEAC4&project=MIZOPEX&project=IFloodS&project=HyMeX&project=PODEX&project=DC3&project=BROMEX&project=GCPEx&project=DEVOTE&project=COAST&project=ECO-3&project=ICESCAPE&project=MC3&project=MACPEX&project=LPVEx&project=GRIP&project=CASIE&project=AMISA&project=ARCTAS&project=TC4&project=CLASIC07&project=C3&project=NAMMA&project=CC-VEx&project=INTEX-NA&project=CR-AVE&project=TCSP&project=BigFoot&project=ACES&project=CRYSTAL-FACE&project=CAMEX&project=PAC2001&project=CLAMS&project=TRACE&project=SAFARI&project=ARESE&project=PEM&project=TEFLUN&project=FIRE&project=POLARIS&project=PROVE&project=BOREAS&project=TARFOX&project=ASHOE&project=SPADE&project=AASE&project=OTTER&project=ABLE&project=CITE&project=FIFE&project=AAOE&project=SNF"


This is an absolute maintenance nightmare. Tags feel much more useful than a list of projects. But if we absolutely must do a list of projects then the config file needs to be an actual list of projects, not a very long string of url parameters.

eudoroolivares2016 · 2023-12-06T14:53:19Z

portals/casei/config.json

+  "parentConfig": "edsc",
+  "portalBrowser": true,
+  "query": {
+    "project": "ABoVE&project=SnowEx&project=Delta-X&project=S-MODE&project=IMPACTS&project=SASSIE&project=CPEX-CV&project=ACCLIP&project=CALIPSO-NVF&project=DCOTSS&project=ACTIVATE&project=MOOSE&project=CPEX-AW&project=TRACER-AQ&project=OMG&project=EXPORTS&project=CAMP2&project=FIREX-AQ&project=LISTOS&project=ACT-America&project=SCOAPE&project=Aeolus&project=ORACLES&project=HyspIRI&project=OWLETS&project=ATom&project=NAAMES&project=ICE-POP&project=SPURS&project=ACEPOL&project=EPOCH&project=ASCENDS&project=CPEX&project=LMOS&project=CORAL&project=CARAFE&project=GOES-R&project=POSIDON&project=KORUS-AQ&project=KORUS-OC&project=OLYMPEX&project=AfriSAR&project=RADEX&project=CARVE&project=VIRGAS&project=AirMOSS&project=COMEX&project=Polar&project=ATTREX&project=CalWater&project=HS3&project=ARISE&project=DISCOVER-AQ&project=SABOR&project=IPHEx&project=SEAC4&project=MIZOPEX&project=IFloodS&project=HyMeX&project=PODEX&project=DC3&project=BROMEX&project=GCPEx&project=DEVOTE&project=COAST&project=ECO-3&project=ICESCAPE&project=MC3&project=MACPEX&project=LPVEx&project=GRIP&project=CASIE&project=AMISA&project=ARCTAS&project=TC4&project=CLASIC07&project=C3&project=NAMMA&project=CC-VEx&project=INTEX-NA&project=CR-AVE&project=TCSP&project=BigFoot&project=ACES&project=CRYSTAL-FACE&project=CAMEX&project=PAC2001&project=CLAMS&project=TRACE&project=SAFARI&project=ARESE&project=PEM&project=TEFLUN&project=FIRE&project=POLARIS&project=PROVE&project=BOREAS&project=TARFOX&project=ASHOE&project=SPADE&project=AASE&project=OTTER&project=ABLE&project=CITE&project=FIFE&project=AAOE&project=SNF"


Check if you can do this in the config

Suggested change

"project": "ABoVE&project=SnowEx&project=Delta-X&project=S-MODE&project=IMPACTS&project=SASSIE&project=CPEX-CV&project=ACCLIP&project=CALIPSO-NVF&project=DCOTSS&project=ACTIVATE&project=MOOSE&project=CPEX-AW&project=TRACER-AQ&project=OMG&project=EXPORTS&project=CAMP2&project=FIREX-AQ&project=LISTOS&project=ACT-America&project=SCOAPE&project=Aeolus&project=ORACLES&project=HyspIRI&project=OWLETS&project=ATom&project=NAAMES&project=ICE-POP&project=SPURS&project=ACEPOL&project=EPOCH&project=ASCENDS&project=CPEX&project=LMOS&project=CORAL&project=CARAFE&project=GOES-R&project=POSIDON&project=KORUS-AQ&project=KORUS-OC&project=OLYMPEX&project=AfriSAR&project=RADEX&project=CARVE&project=VIRGAS&project=AirMOSS&project=COMEX&project=Polar&project=ATTREX&project=CalWater&project=HS3&project=ARISE&project=DISCOVER-AQ&project=SABOR&project=IPHEx&project=SEAC4&project=MIZOPEX&project=IFloodS&project=HyMeX&project=PODEX&project=DC3&project=BROMEX&project=GCPEx&project=DEVOTE&project=COAST&project=ECO-3&project=ICESCAPE&project=MC3&project=MACPEX&project=LPVEx&project=GRIP&project=CASIE&project=AMISA&project=ARCTAS&project=TC4&project=CLASIC07&project=C3&project=NAMMA&project=CC-VEx&project=INTEX-NA&project=CR-AVE&project=TCSP&project=BigFoot&project=ACES&project=CRYSTAL-FACE&project=CAMEX&project=PAC2001&project=CLAMS&project=TRACE&project=SAFARI&project=ARESE&project=PEM&project=TEFLUN&project=FIRE&project=POLARIS&project=PROVE&project=BOREAS&project=TARFOX&project=ASHOE&project=SPADE&project=AASE&project=OTTER&project=ABLE&project=CITE&project=FIFE&project=AAOE&project=SNF"

"project": "ABoVE",

"project":"SnowEx"

...

unless we customize some logic from the docs it didn't seem like CMR could parse project=['proj1', 'proj2] etc

Nope, EDSC just keeps the last line if there are multiple lines of "project" keys.
An array would be a nice compromise and might be worth altering parsing logic for.

Why isn't this using tags?

We'll discuss with the product owner next week whether a tags-based workflow would be better for the data owners and users.

In the meantime, the latest commit changes the project list to a string array in portals/casei/config.json, and adds project to the array of keys in sharedConstants/nonIndexedCmrKeys.js. That allows it to be stringified correctly by sharedUtils/prepKeysForCmr.js.

eudoroolivares2016 · 2023-12-12T16:09:14Z

After discussing the context was that we don't expect these projects to change. This is a stop-gap measure until the UMM-C changes (or perhaps other determined CMR changes) can be made to accommodate casei field data at which point likely other measures (such as a new facet) may be used to filter the collections

Bccorb · 2023-12-14T18:17:34Z

static/src/js/util/request/__tests__/collectionRequest.test.js

@@ -104,6 +104,7 @@ describe('CollectionRequest#nonIndexedKeys', () => {
      'point',
      'polygon',
      'processing_level_id_h',
+      'project',


QQ: Did project_h not work as the query param to process? Instead of adding project?

eudoroolivares2016

Lets fix the project facet or determine if we think that warrants holding this off

eudoroolivares2016

We discussed that facet issue it is something existing so that should not stop this from completing

Doesn't work. The way the "condition" and "or" keys are stringified are incompatible with CMR. The stringified array indices for projects are also unneccessary.

macrouch reviewed Dec 6, 2023

View reviewed changes

eudoroolivares2016 reviewed Dec 6, 2023

View reviewed changes

rushgeo force-pushed the EDSC-3881 branch from 3c59171 to 46832ad Compare December 12, 2023 15:38

rushgeo marked this pull request as ready for review December 12, 2023 15:38

eudoroolivares2016 approved these changes Dec 12, 2023

View reviewed changes

rushgeo requested review from Bccorb, bnp26, dpesall, daniel-zamora and macrouch December 12, 2023 16:22

Bccorb reviewed Dec 14, 2023

View reviewed changes

Bccorb self-requested a review December 14, 2023 19:17

Bccorb approved these changes Dec 14, 2023

View reviewed changes

eudoroolivares2016 requested changes Dec 14, 2023

View reviewed changes

eudoroolivares2016 self-requested a review December 15, 2023 15:36

eudoroolivares2016 approved these changes Dec 15, 2023

View reviewed changes

rushgeo force-pushed the EDSC-3881 branch from 46832ad to b03b334 Compare December 15, 2023 18:36

rushgeo added 4 commits December 15, 2023 15:28

EDSC-3881: Create portal for CASEI

b204709

EDSC-3881: WIP, attempt to use object array

efb599c

Doesn't work. The way the "condition" and "or" keys are stringified are incompatible with CMR. The stringified array indices for projects are also unneccessary.

EDSC-3881: Use a string array for projects

e7c3bf2

EDSC-3881: fix Jest test

fb5838c

rushgeo force-pushed the EDSC-3881 branch from b03b334 to fb5838c Compare December 15, 2023 20:28

rushgeo merged commit d5403eb into main Dec 15, 2023
9 checks passed

rushgeo deleted the EDSC-3881 branch December 15, 2023 21:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EDSC-3881: Create portal for CASEI #1699

EDSC-3881: Create portal for CASEI #1699

rushgeo commented Dec 5, 2023 •

edited

Loading

codecov bot commented Dec 5, 2023 •

edited

Loading

macrouch Dec 6, 2023

eudoroolivares2016 Dec 6, 2023

rushgeo Dec 6, 2023

macrouch Dec 6, 2023

rushgeo Dec 8, 2023

eudoroolivares2016 commented Dec 12, 2023

Bccorb Dec 14, 2023

eudoroolivares2016 left a comment

eudoroolivares2016 left a comment

	"project": "ABoVE&project=SnowEx&project=Delta-X&project=S-MODE&project=IMPACTS&project=SASSIE&project=CPEX-CV&project=ACCLIP&project=CALIPSO-NVF&project=DCOTSS&project=ACTIVATE&project=MOOSE&project=CPEX-AW&project=TRACER-AQ&project=OMG&project=EXPORTS&project=CAMP2&project=FIREX-AQ&project=LISTOS&project=ACT-America&project=SCOAPE&project=Aeolus&project=ORACLES&project=HyspIRI&project=OWLETS&project=ATom&project=NAAMES&project=ICE-POP&project=SPURS&project=ACEPOL&project=EPOCH&project=ASCENDS&project=CPEX&project=LMOS&project=CORAL&project=CARAFE&project=GOES-R&project=POSIDON&project=KORUS-AQ&project=KORUS-OC&project=OLYMPEX&project=AfriSAR&project=RADEX&project=CARVE&project=VIRGAS&project=AirMOSS&project=COMEX&project=Polar&project=ATTREX&project=CalWater&project=HS3&project=ARISE&project=DISCOVER-AQ&project=SABOR&project=IPHEx&project=SEAC4&project=MIZOPEX&project=IFloodS&project=HyMeX&project=PODEX&project=DC3&project=BROMEX&project=GCPEx&project=DEVOTE&project=COAST&project=ECO-3&project=ICESCAPE&project=MC3&project=MACPEX&project=LPVEx&project=GRIP&project=CASIE&project=AMISA&project=ARCTAS&project=TC4&project=CLASIC07&project=C3&project=NAMMA&project=CC-VEx&project=INTEX-NA&project=CR-AVE&project=TCSP&project=BigFoot&project=ACES&project=CRYSTAL-FACE&project=CAMEX&project=PAC2001&project=CLAMS&project=TRACE&project=SAFARI&project=ARESE&project=PEM&project=TEFLUN&project=FIRE&project=POLARIS&project=PROVE&project=BOREAS&project=TARFOX&project=ASHOE&project=SPADE&project=AASE&project=OTTER&project=ABLE&project=CITE&project=FIFE&project=AAOE&project=SNF"
	"project": "ABoVE",
	"project":"SnowEx"

EDSC-3881: Create portal for CASEI #1699

EDSC-3881: Create portal for CASEI #1699

Conversation

rushgeo commented Dec 5, 2023 • edited Loading

Overview

What is the feature?

What is the Solution?

Testing

Attachments

Checklist

codecov bot commented Dec 5, 2023 • edited Loading

Codecov Report

macrouch Dec 6, 2023

Choose a reason for hiding this comment

eudoroolivares2016 Dec 6, 2023

Choose a reason for hiding this comment

rushgeo Dec 6, 2023

Choose a reason for hiding this comment

macrouch Dec 6, 2023

Choose a reason for hiding this comment

rushgeo Dec 8, 2023

Choose a reason for hiding this comment

eudoroolivares2016 commented Dec 12, 2023

Bccorb Dec 14, 2023

Choose a reason for hiding this comment

eudoroolivares2016 left a comment

Choose a reason for hiding this comment

eudoroolivares2016 left a comment

Choose a reason for hiding this comment

rushgeo commented Dec 5, 2023 •

edited

Loading

codecov bot commented Dec 5, 2023 •

edited

Loading