NDCube, Scipp, Xarray, and developments in the Python world

[SimonHeybrock]

Hi!
I am one of the developers of Scipp, and we just came across your project. Scipp is developed for processing neutron-scattering data at the ESS. Due to a lot of similarities with NDCube (labeled ND-data, units, plotting, masking, uncertainties, resampling, ...) and certain choices (such as not using Xarray) I thought it could be interesting to get in touch.

Due to active developments in the Python ecosystem, in particular a number of NumPy NEPs (such as duck-arrays, a new dtype system that could support units, ...) as well as Xarray support for custom indexes, we are currently reevaluating our design. Specifically, I am currently looking into whether a number of features we require could be implemented in simple and independent "duck arrays" that could be used, e.g., with Xarray (or another, simpler, DataArray implementation). If feasible, this could open opportunities for more collaboration on smaller building blocks as those should be less subject to the typically highly specific requirements/needs in the various scientific communities.

In any case, I would be very interested in hearing a bit more about NDCube's background and your view of the "world".

Simon

[Cadair]

hey @SimonHeybrock sorry for not getting back to you I looked at Scipp and then completely forgot to write you a response 🤦.

NDCube's design is driven by our functional coordinate systems see this paragraph for a quick description. We are also in the early stages of people experimenting with the xarray indicies see #222.

It certainly looks like some of the stuff in scipp would be applicable to us, and definitely would be something we evaluated if we started over now.

Thanks for reaching out and let us know if there's anything else you would like to know / collaborate on!