This gitlab repository contains python scripts for getting and processing SDO/AIA, SDO/HMI, and SDO/EVE files to new cloud-friendly .zarr files (SDOMLv2). The data is essentially the same as previous SDOML. For more details about the SDOML dataset, please refer to the following paper:
"A Machine-learning Data Set Prepared from the NASA Solar Dynamics Observatory Mission" http://adsabs.harvard.edu/abs/2019ApJS..242....7G
The major changes in SDOMLv2 include:
- Using the cloud-friendly Zarr format for the storage of chunked, compressed, N-dimensional arrays (https://zarr.readthedocs.io/en/stable/).
- All fits header information is saved in the meta data.
A brief description about each script:
- aia_fits_to_zarr.py for processing the AIA synoptic fits data to zarr files
- hmi_fits_to_zarr.py for processing the HMI fits data to zarr files
- eve_npy_to_zarr.py for processing the EVE npy files (from SDOMLv1) to zarr files
- save_filelists.py for saving file lists of AIA channels