Feature: Padding/Buffer around chunks

[four43]

When querying chunks of data, we will often interpolate beyond the bounds of a single chunk. In order to optimize reading we would like to include data beyond the border of a single chunk into the next. This would require duplicating data. The goal would be to perform a given read, on the "edge" of a chunk, in a single chunk range read. We have used a similar approach in vector tiles/buffering with Mapnik rendering.

Would love to hear thoughts and ideas - thanks!

EDIT - Related ticket in the python project: zarr-developers/zarr-python#1635

[rabernat]

Great idea. We use this approach heavily in CFD.

I think this would make a lot of sense as a Zarr extension. Unfortunately the process for creating extensions is currently pretty poorly defined and documented. We (Zarr steering council) are working on fixing that.

In the meantime, I'd encourage you to experiment with an implementation!