Team Planning - Wednesday, October 23rd

[github-actions]

This is a LEAP Data and Computation Team weekly standup 🎉🎉🎉!
This is a way to share feedback from the past cycle, set goals for the next work cycle and coordinate work with others in the group.

Each member should aim to put down 2-3 goals going into the meeting. We then try to discuss how to best align activities around the group to facilitate collaboration as much as possible.

Expecting responses from all members of @leap-stc/data-and-compute.

Copy and paste the template below, and answer questions as you wish!

Response Template

**Thanks I'd like to give 🙌**
- So-and-so helped me out a lot with XXX...
- Thanks for Jo's work on the XXX...

**User Interactions/Concerns**
- So and so found a gnarly bug. I have raised an issue here ...
- We got some awesome feedback on the new ... feature. 

**Cycle Review 🔎**
- I finished implementing ...
- 

**My goals for the two upcoming weeks 💪**
<!-- Aim to post 2-3 items here before the meeting time, and cross out (not delete) items that you will not work on this week.  --> 
- [ ] A goal/project I need to work on in the next two weeks: <link to issues / PRs>
- [ ] An issue/project that needs help/coordination from another member

**Questions for the group ❓**
<!-- Any questions for the whole group? -->

**My availability for this week 🗓**

[norlandrhagen]

Thanks I'd like to give 🙌

• @jbusecke for getting the pgf meeting revived.
• @andersy005 for help with weird NetCDF4 issues over the cloud.

User Interactions/Concerns

• The thredds server for the eNATL dataset was down for a few days, but Aurélie A. was able to get it fixed.

Cycle Review 🔎

• eNATL600m.
  -Pangeo-Forge

  • ability to read refs - merged
  • ability to read Zarr - merged

  Virtualizarr:

  • Read existing references. ie. we can now checkpoint etc. - merged
  • PR to read exiting zarr Add Zarr Reader(s) zarr-developers/VirtualiZarr#262

My goals for the two upcoming weeks 💪

• eNATL 600m to catalog
• rclone beam stage testing

Questions for the group ❓

My availability for this week 🗓

[jbusecke]

Thanks I'd like to give 🙌

• Sammy for the support and engagement during the annual meeting! Great job representing the DCT!
• @norlandrhagen for the revival of the PGF meeting, a really interesting discussion on the future of pangeo forge, and all the perspective, prototyping with regard to data ingestion.
• @andersy005 for the great chats during the annual meeting.

Cycle Review 🔎

• I think the annual meeting was successful, but I was a bit dissapointed by the lack of critical responses. I think folks are still very hestitant to critique and challenge the infrastructure.

My goals for the two upcoming weeks 💪

• All in on storage and CMIP6 for the next cycle
• Beam copy stage with rclone (potentially test this with gcs too, so we can deprecate the 'old' fsspec copy stage)
  • Test Case: Use rclone copy stage here. chirps_feedstock#2
  • PR
• Finish up longstanding PRs, docs, ... for CMIP6 feedstock
• Major Batch reprocessing of existing recipes
• Freeing up space on leap-persistent by moving stuff to the OSN pod.
• Experiment with icechunk, and document how this might fit into our plans here (also consider arraylake)

My availability for this week 🗓

• Normal cycle: Fri Lamont, Mon/Tue WFH, Wed/Thu LEAP.

[andersy005]

Thanks I'd like to give 🙌

• @jbusecke for providing feedback on the catalog revamp/feedstock page: https://catalog.leap.columbia.edu/
• @norlandrhagen for helping with sample datasets to test catalog features

User Interactions/Concerns

Cycle Review 🔎

• I finished implementing
  • add dynamic /feedstock/[slug] page carbonplan/leap-data-catalog#82
  • [Catalog Feature] xarray html repr for each store data-management#140

My goals for the two upcoming weeks 💪

• Can we configure the 'engine' argument in the displayed xarray/python repr? data-management#150
• Analytics for Dataviewer, Stories and Catalog? #16
• Adding more info to catalog cards data-management#136

Questions for the group ❓

My availability for this week 🗓