netCDF4 unable to open netCDF file that opens as HDF and with Panoply #1313

danielfromearth · 2024-03-15T17:07:45Z

The version of the software with which you are encountering an issue

h5netcdf version: 1.2.0
libnetcdf version: 4.9.2
netCDF4 version: 1.6.5
xarray version: 2024.2.0

Environmental information (i.e. Operating System, compiler info, java version, python version, etc.)

OS: MacOS 14.3.1
Chip: Apple M3 Pro
Python version: 3.11.8

A description of the issue with the steps needed to reproduce it

The following error is raised when attempting to open the file (linked below) via the netCDF4 python library:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[10], line 1
----> 1 nc.Dataset(filepath)

File src[/netCDF4/_netCDF4.pyx:2495](http://localhost:8888/netCDF4/_netCDF4.pyx#line=2494), in netCDF4._netCDF4.Dataset.__init__()

File src[/netCDF4/_netCDF4.pyx:1891](http://localhost:8888/netCDF4/_netCDF4.pyx#line=1890), in netCDF4._netCDF4._get_grps()

File src[/netCDF4/_netCDF4.pyx:3616](http://localhost:8888/netCDF4/_netCDF4.pyx#line=3615), in netCDF4._netCDF4.Group.__init__()

File src[/netCDF4/_netCDF4.pyx:2001](http://localhost:8888/netCDF4/_netCDF4.pyx#line=2000), in netCDF4._netCDF4._get_vars()

AttributeError: 'NoneType' object has no attribute 'dimensions'

Here is where the file can be downloaded. It does require a free NASA Earthdata login account to access, but the file size even when compressed (51 MB) exceeds the GitHub issue limit of 25 MB so I couldn't upload it here.

Interestingly, as an HDF5, the file appears OK: The whole file can be opened without error by the underlying library libhdf5, as well as h5dump, h5netcdf, and the myHDF5 tool. However, the irradiance and irradiance_error variables — which exist in multiple groups, e.g., /frames/band_290_490_nm/ — do raise errors when the HDFView application attempts to open them.

Also, interestingly, the NASA Panoply data viewer opens the file without issue as well.

The text was updated successfully, but these errors were encountered:

DennisHeimbigner · 2024-03-15T17:28:36Z

Are you using libnetcdf? If so, which version.

danielfromearth · 2024-03-15T17:30:21Z

I'm in a conda environment, which has libnetcdf version 4.9.2 installed.

jswhit2 · 2024-03-19T21:40:30Z

is ncdump able to read this file?

dopplershift · 2024-03-19T21:47:15Z

I can reproduce this problem (on my Mac M2) in Python 3.12:

# Name                    Version                   Build  Channel
libnetcdf                 4.9.2           nompi_h291a7c2_113    conda-forge
netcdf4                   1.6.5           nompi_py312h9035142_100    conda-forge

While ncdump -h works without error, this was interesting:

❯ ncdump TEMPO_IRR_L1_V02_20240314T041808Z.nc > /dev/null
NetCDF: Start+count exceeds dimension bound
Location: file ?; fcn ? line 478

dopplershift · 2024-03-19T21:55:22Z

ncdump -sh output if anyone is curious:

netcdf TEMPO_IRR_L1_V02_20240314T041808Z {
dimensions:
	mirror_step = 1 ;
variables:
	int mirror_step(mirror_step) ;
		mirror_step:long_name = "scan mirror position index" ;
		mirror_step:_Storage = "contiguous" ;
		mirror_step:_Endianness = "little" ;
	double time(mirror_step) ;
		time:standard_name = "time" ;
		time:long_name = "exposure start time" ;
		time:calendar = "gregorian" ;
		time:units = "seconds since 1980-01-06T00:00:00Z" ;
		time:comment = "Average of /frames/time" ;
		time:_Storage = "contiguous" ;
		time:_Endianness = "little" ;
	float exposure_time(mirror_step) ;
		exposure_time:units = "seconds" ;
		exposure_time:long_name = "exposure duration" ;
		exposure_time:valid_min = 0.f ;
		exposure_time:valid_max = 10.f ;
		exposure_time:_FillValue = 9.96921e+36f ;
		exposure_time:comment = "Average of /frames/exposure_time" ;
		exposure_time:_Storage = "contiguous" ;
		exposure_time:_Endianness = "little" ;
	float solar_phi(mirror_step) ;
		solar_phi:units = "degrees" ;
		solar_phi:long_name = "solar boresight azimuthal angle" ;
		solar_phi:valid_min = -180.f ;
		solar_phi:valid_max = 180.f ;
		solar_phi:_FillValue = 9.96921e+36f ;
		solar_phi:comment = "Average of /frames/solar_phi" ;
		solar_phi:_Storage = "contiguous" ;
		solar_phi:_Endianness = "little" ;
	float solar_theta(mirror_step) ;
		solar_theta:units = "degrees" ;
		solar_theta:long_name = "solar boresight polar angle" ;
		solar_theta:valid_min = 0.f ;
		solar_theta:valid_max = 180.f ;
		solar_theta:_FillValue = 9.96921e+36f ;
		solar_theta:comment = "Average of /frames/solar_theta" ;
		solar_theta:_Storage = "contiguous" ;
		solar_theta:_Endianness = "little" ;
	float earth_sun_distance ;
		earth_sun_distance:units = "m" ;
		earth_sun_distance:long_name = "Earth-sun distance" ;
		earth_sun_distance:_Storage = "contiguous" ;
		earth_sun_distance:_Endianness = "little" ;
// global attributes:

:time_reference = "1980-01-06T00:00:00Z" ;

:format_version = 2 ;

:product_type = "IRR" ;

:processing_level = "1" ;

:sdpc_version = "TEMPO_SDPC_v4.3.0" ;

:processing_version = 2 ;

:production_date_time = "2024-03-14T04:39:37Z" ;

:local_granule_id = "TEMPO_IRR_L1_V02_20240314T041808Z.nc" ;

:version_id = 2 ;

:pge_version = "0.1.0" ;

:shortname = "TEMPO_IRR_L1" ;

:begin_date = "2024-03-14" ;

:begin_time = "04:18:08" ;

:end_date = "2024-03-14" ;

:end_time = "04:18:11" ;

:time_coverage_start = "2024-03-14T04:18:08Z" ;

:time_coverage_end = "2024-03-14T04:18:11Z" ;

:time_coverage_start_since_epoch = 1394425106.5535 ;

:time_coverage_end_since_epoch = 1394425109.21705 ;

:history = "2024-03-14T04:39:44Z: /tempo/nas0/sdpc_soft/install/gnu-rh8/sdpc/v4p3/bin/L0_ccd -vv --Version 2 --config l0_ccd.cfg -i @hk.lis --trend trend_params.nc -d /tempo/nas0/sdpc_archive/temposdpc/ops3/L1/DRK/D16138/20240314T041121Z/TEMPO_DRK_L1_V02_20240314T041121Z.nc -o TEMPO_IRR_L1_V02_20240314T041808Z.nc TEMPO_IRR_L0_V01_20240314T041808Z.nc\n2024-03-14T04:57:25Z: /tempo/nas0/sdpc_soft/install/gnu-rh8/sdpc/v4p3/bin/wavecal_merge --meta --delete -t TEMPO_IRR_L1_V02_20240314T041808Z.nc wavecal_TEMPO_IRR_L1_V02_20240314T041808Z\n" ;

string :input_files = "TEMPO_IRR_L0_V01_20240314T041808Z.nc", "TEMPO_HK_L0_V01_20240314T040946Z.nc", "TEMPO_HK_L0_V01_20240314T041410Z.nc", "TEMPO_HK_L0_V01_20240314T041840Z.nc", "TEMPO_HK_L0_V01_20240314T042310Z.nc", "TEMPO_DRK_L1_V02_20240314T041121Z.nc" ;

:Conventions = "CF-1.6, ACDD-1.3" ;

:day_of_year = 74LL ;

:project = "TEMPO" ;

:platform = "Intelsat 40e" ;

:source = "UV-VIS hyperspectral imaging" ;

:institution = "Smithsonian Astrophysical Observatory" ;

:creator_url = "http://tempo.si.edu" ;

:title = "TEMPO Level 1 irradiance measurement" ;

:collection_shortname = "TEMPO_IRR_L1" ;

:collection_version = 1LL ;

:keywords = "EARTH SCIENCE>ATMOSPHERE>ATMOSPHERIC RADIATION>SOLAR IRRADIANCE" ;

:summary = "Level 1 irradiance files provide solar irradiance measured using the working solar diffuser. Each file includes the measured solar irradiance for all the North-South cross-track pixels. The files are provided in netCDF4 format, and contain information on radiometrically and wavelength calibrated solar irradiance for the UV and visible bands, corresponding noise, parameterized wavelength grid, solar viewing geometry, quality flags and other ancillary information. The product is produced using the L0-1b processor which includes multiple steps: (1) Image processing to produce radiometrically calibrated radiance, and (2) Additional wavelength calibration to improve wavelength registration. Please refer to the ATBD for details." ;

:coremetadata = "\nGROUP                  = INVENTORYMETADATA\n  GROUPTYPE            = MASTERGROUP\n\n  GROUP                  = ECSDATAGRANULE\n\n    OBJECT                 = LOCALGRANULEID\n      NUM_VAL              = 1\n      VALUE                = "TEMPO_IRR_L1_V02_20240314T041808Z.nc"\n    END_OBJECT             = LOCALGRANULEID\n\n    OBJECT                 = LOCALVERSIONID\n      NUM_VAL              = 1\n      VALUE                = ("RFC1321 MD5 = not yet calculated")\n    END_OBJECT             = LOCALVERSIONID\n\n    OBJECT                 = PRODUCTIONDATETIME\n      NUM_VAL              = 1\n      VALUE                = "2024-03-14T04:39:37Z"\n    END_OBJECT             = PRODUCTIONDATETIME\n\n  END_GROUP              = ECSDATAGRANULE\n\n  GROUP                  = COLLECTIONDESCRIPTIONCLASS\n\n    OBJECT                 = SHORTNAME\n      NUM_VAL              = 1\n      VALUE                = "TEMPO_IRR_L1"\n    END_OBJECT             = SHORTNAME\n\n    OBJECT                 = VERSIONID\n      NUM_VAL              = 1\n      VALUE                = 2\n    END_OBJECT             = VERSIONID\n\n  END_GROUP              = COLLECTIONDESCRIPTIONCLASS\n\n  GROUP                  = INPUTGRANULE\n\n    OBJECT                 = INPUTPOINTER\n      NUM_VAL              = 6\n      VALUE                = ("TEMPO_IRR_L0_V01_20240314T041808Z.nc", "TEMPO_HK_L0_V01_20240314T040946Z.nc", "TEMPO_HK_L0_V01_20240314T041410Z.nc", "TEMPO_HK_L0_V01_20240314T041840Z.nc", "TEMPO_HK_L0_V01_20240314T042310Z.nc", "TEMPO_DRK_L1_V02_20240314T041121Z.nc")\n    END_OBJECT             = INPUTPOINTER\n\n  END_GROUP              = INPUTGRANULE\n\n  GROUP = SPATIALDOMAINCONTAINER\n    OBJECT = GRANULESPATIALREPRESENTATION\n      CLASS = "1"\n      NUM_VAL = 1\n      VALUE = "NO_SPATIAL"\n    END_OBJECT = GRANULESPATIALREPRESENTATION\n  END_GROUP = SPATIALDOMAINCONTAINER\n\n  GROUP                  = RANGEDATETIME\n\n    OBJECT                 = RANGEENDINGDATE\n      NUM_VAL              = 1\n      VALUE                = "2024-03-14"\n    END_OBJECT             = RANGEENDINGDATE\n\n    OBJECT                 = RANGEENDINGTIME\n      NUM_VAL              = 1\n      VALUE                = "04:18:11"\n    END_OBJECT             = RANGEENDINGTIME\n\n    OBJECT                 = RANGEBEGINNINGDATE\n      NUM_VAL              = 1\n      VALUE                = "2024-03-14"\n    END_OBJECT             = RANGEBEGINNINGDATE\n\n    OBJECT                 = RANGEBEGINNINGTIME\n      NUM_VAL              = 1\n      VALUE                = "04:18:08"\n    END_OBJECT             = RANGEBEGINNINGTIME\n\n  END_GROUP              = RANGEDATETIME\n\n  GROUP                  = PGEVERSIONCLASS\n\n    OBJECT                 = PGEVERSION\n      NUM_VAL              = 1\n      VALUE                = "0.1.0"\n    END_OBJECT             = PGEVERSION\n\n  END_GROUP              = PGEVERSIONCLASS\n\nEND_GROUP              = INVENTORYMETADATA\n\nEND\n" ;

:_NCProperties = "version=1|netcdflibversion=4.6.1|hdf5libversion=1.8.14" ;

:_SuperblockVersion = 0 ;

:_IsNetcdf4 = 1 ;

:_Format = "netCDF-4" ;
group: band_290_490_nm {

dimensions:

xtrack = 2048 ;

spectral_channel = 1028 ;

xtrack = 2048 ;

variables:

int xtrack(xtrack) ;

xtrack:_Storage = "contiguous" ;

xtrack:_Endianness = "little" ;

float irradiance(mirror_step, xtrack, spectral_channel) ;

irradiance:units = "photons/s/cm^2/nm" ;

irradiance:ancillary_variables = "irradiance_error" ;

irradiance:_FillValue = -9.96921e+36f ;

irradiance:comment = "Average of /frames/band_290_490_nm/irradiance" ;

irradiance:_Storage = "chunked" ;

irradiance:_ChunkSizes = 1, 128, 1028 ;

irradiance:_Shuffle = "true" ;

irradiance:_DeflateLevel = 1 ;

irradiance:_Endianness = "little" ;

float irradiance_error(mirror_step, xtrack, spectral_channel) ;

irradiance_error:units = "photons/s/cm^2/nm" ;

irradiance_error:long_name = "irradiance error" ;

irradiance_error:_FillValue = -9.96921e+36f ;

irradiance_error:number_of_significant_digits = 2 ;

irradiance_error:comment = "Root mean square of /frames/band_290_490_nm/irradiance_error" ;

irradiance_error:_Storage = "chunked" ;

irradiance_error:_ChunkSizes = 1, 128, 1028 ;

irradiance_error:_Shuffle = "true" ;

irradiance_error:_DeflateLevel = 1 ;

irradiance_error:_Endianness = "little" ;

float nominal_wavelength(xtrack, spectral_channel) ;

nominal_wavelength:units = "nm" ;

nominal_wavelength:valid_min = -9.96921e+36f ;

nominal_wavelength:valid_max = 9.96921e+36f ;

nominal_wavelength:_FillValue = 9.96921e+36f ;

nominal_wavelength:_Storage = "contiguous" ;

nominal_wavelength:_Endianness = "little" ;

ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;

pixel_quality_flag:long_name = "pixel quality flag" ;

pixel_quality_flag:_FillValue = 65535US ;

pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;

pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;

pixel_quality_flag:comment = "Logical OR of /frames/band_290_490_nm/pixel_quality_flag" ;

pixel_quality_flag:_Storage = "chunked" ;

pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;

pixel_quality_flag:_Shuffle = "true" ;

pixel_quality_flag:_DeflateLevel = 1 ;

pixel_quality_flag:_Endianness = "little" ;

float wavecal_params(mirror_step, xtrack, /xtrack) ;

wavecal_params:num_coefficients = 2 ;

wavecal_params:start_spectral_channel = 0 ;

wavecal_params:num_spectral_channels = 1028 ;

wavecal_params:_Storage = "contiguous" ;

wavecal_params:_Endianness = "little" ;

float sf_hw1e(mirror_step, xtrack, spectral_channel) ;

sf_hw1e:_Storage = "contiguous" ;

sf_hw1e:_Endianness = "little" ;

float sf_shape(mirror_step, xtrack, spectral_channel) ;

sf_shape:_Storage = "contiguous" ;

sf_shape:_Endianness = "little" ;

float sf_asym(mirror_step, xtrack, spectral_channel) ;

sf_asym:_Storage = "contiguous" ;

sf_asym:_Endianness = "little" ;

int wavecal_niter(mirror_step, xtrack) ;

wavecal_niter:_Storage = "contiguous" ;

wavecal_niter:_Endianness = "little" ;

int wavecal_opt_status(mirror_step, xtrack) ;

wavecal_opt_status:_Storage = "contiguous" ;

wavecal_opt_status:_Endianness = "little" ;
// group attributes:

} // group band_290_490_nm
group: band_540_740_nm {

dimensions:

xtrack = 2048 ;

spectral_channel = 1028 ;

mirror_step = 1 ;

variables:

int xtrack(xtrack) ;

xtrack:_Storage = "contiguous" ;

xtrack:_Endianness = "little" ;

float irradiance(mirror_step, xtrack, spectral_channel) ;

irradiance:units = "photons/s/cm^2/nm" ;

irradiance:ancillary_variables = "irradiance_error" ;

irradiance:_FillValue = -9.96921e+36f ;

irradiance:comment = "Average of /frames/band_540_740_nm/irradiance" ;

irradiance:_Storage = "chunked" ;

irradiance:_ChunkSizes = 1, 128, 1028 ;

irradiance:_Shuffle = "true" ;

irradiance:_DeflateLevel = 1 ;

irradiance:_Endianness = "little" ;

float irradiance_error(mirror_step, xtrack, spectral_channel) ;

irradiance_error:units = "photons/s/cm^2/nm" ;

irradiance_error:long_name = "irradiance error" ;

irradiance_error:_FillValue = -9.96921e+36f ;

irradiance_error:number_of_significant_digits = 2 ;

irradiance_error:comment = "Root mean square of /frames/band_540_740_nm/irradiance_error" ;

irradiance_error:_Storage = "chunked" ;

irradiance_error:_ChunkSizes = 1, 128, 1028 ;

irradiance_error:_Shuffle = "true" ;

irradiance_error:_DeflateLevel = 1 ;

irradiance_error:_Endianness = "little" ;

float nominal_wavelength(xtrack, spectral_channel) ;

nominal_wavelength:units = "nm" ;

nominal_wavelength:valid_min = -9.96921e+36f ;

nominal_wavelength:valid_max = 9.96921e+36f ;

nominal_wavelength:_FillValue = 9.96921e+36f ;

nominal_wavelength:_Storage = "contiguous" ;

nominal_wavelength:_Endianness = "little" ;

ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;

pixel_quality_flag:long_name = "pixel quality flag" ;

pixel_quality_flag:_FillValue = 65535US ;

pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;

pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;

pixel_quality_flag:comment = "Logical OR of /frames/band_540_740_nm/pixel_quality_flag" ;

pixel_quality_flag:_Storage = "chunked" ;

pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;

pixel_quality_flag:_Shuffle = "true" ;

pixel_quality_flag:_DeflateLevel = 1 ;

pixel_quality_flag:_Endianness = "little" ;

float wavecal_params(mirror_step, xtrack, /mirror_step) ;

wavecal_params:num_coefficients = 3 ;

wavecal_params:start_spectral_channel = 0 ;

wavecal_params:num_spectral_channels = 1028 ;

wavecal_params:_Storage = "contiguous" ;

wavecal_params:_Endianness = "little" ;

float sf_hw1e(mirror_step, xtrack, spectral_channel) ;

sf_hw1e:_Storage = "contiguous" ;

sf_hw1e:_Endianness = "little" ;

float sf_shape(mirror_step, xtrack, spectral_channel) ;

sf_shape:_Storage = "contiguous" ;

sf_shape:_Endianness = "little" ;

float sf_asym(mirror_step, xtrack, spectral_channel) ;

sf_asym:_Storage = "contiguous" ;

sf_asym:_Endianness = "little" ;

int wavecal_niter(mirror_step, xtrack) ;

wavecal_niter:_Storage = "contiguous" ;

wavecal_niter:_Endianness = "little" ;

int wavecal_opt_status(mirror_step, xtrack) ;

wavecal_opt_status:_Storage = "contiguous" ;

wavecal_opt_status:_Endianness = "little" ;
// group attributes:

} // group band_540_740_nm
group: frames {

dimensions:

mirror_step = 1 ;

variables:

int mirror_step(mirror_step) ;

mirror_step:long_name = "scan mirror position index" ;

mirror_step:_Storage = "contiguous" ;

mirror_step:_Endianness = "little" ;

double time(mirror_step) ;

time:standard_name = "time" ;

time:long_name = "exposure start time" ;

time:calendar = "gregorian" ;

time:units = "seconds since 1980-01-06T00:00:00Z" ;

time:_Storage = "contiguous" ;

time:_Endianness = "little" ;

float exposure_time(mirror_step) ;

exposure_time:units = "seconds" ;

exposure_time:long_name = "exposure duration" ;

exposure_time:valid_min = 0.f ;

exposure_time:valid_max = 10.f ;

exposure_time:_FillValue = 9.96921e+36f ;

exposure_time:_Storage = "contiguous" ;

exposure_time:_Endianness = "little" ;

float solar_phi(mirror_step) ;

solar_phi:units = "degrees" ;

solar_phi:long_name = "solar boresight azimuthal angle" ;

solar_phi:valid_min = -180.f ;

solar_phi:valid_max = 180.f ;

solar_phi:_FillValue = 9.96921e+36f ;

solar_phi:_Storage = "contiguous" ;

solar_phi:_Endianness = "little" ;

float solar_theta(mirror_step) ;

solar_theta:units = "degrees" ;

solar_theta:long_name = "solar boresight polar angle" ;

solar_theta:valid_min = 0.f ;

solar_theta:valid_max = 180.f ;

solar_theta:_FillValue = 9.96921e+36f ;

solar_theta:_Storage = "contiguous" ;

solar_theta:_Endianness = "little" ;

float earth_sun_distance ;

earth_sun_distance:units = "m" ;

earth_sun_distance:long_name = "Earth-sun distance" ;

earth_sun_distance:_Storage = "contiguous" ;

earth_sun_distance:_Endianness = "little" ;
// group attributes:

:time_reference = "1980-01-06T00:00:00Z" ;

:Conventions = "CF-1.6, ACDD-1.3" ;

:format_version = 2 ;

:product_type = "IRR" ;

:processing_level = "1" ;

:sdpc_version = "TEMPO_SDPC_v4.3.0" ;

:time_coverage_start = "2024-03-14T04:18:08Z" ;

:time_coverage_end = "2024-03-14T04:18:11Z" ;

:processing_version = 2 ;

:time_coverage_start_since_epoch = 1394425106.5535 ;

:time_coverage_end_since_epoch = 1394425109.21705 ;
group: band_290_490_nm {

dimensions:

xtrack = 2048 ;

spectral_channel = 1028 ;

variables:

int xtrack(xtrack) ;

xtrack:_Storage = "contiguous" ;

xtrack:_Endianness = "little" ;

float irradiance(mirror_step, xtrack, spectral_channel) ;

irradiance:units = "photons/s/cm^2/nm" ;

irradiance:ancillary_variables = "irradiance_error" ;

irradiance:_FillValue = -9.96921e+36f ;

irradiance:_Storage = "chunked" ;

irradiance:_ChunkSizes = 1, 128, 1028 ;

irradiance:_Shuffle = "true" ;

irradiance:_DeflateLevel = 1 ;

irradiance:_Endianness = "little" ;

float irradiance_error(mirror_step, xtrack, spectral_channel) ;

irradiance_error:units = "photons/s/cm^2/nm" ;

irradiance_error:long_name = "irradiance error" ;

irradiance_error:_FillValue = -9.96921e+36f ;

irradiance_error:number_of_significant_digits = 2 ;

irradiance_error:_Storage = "chunked" ;

irradiance_error:_ChunkSizes = 1, 128, 1028 ;

irradiance_error:_Shuffle = "true" ;

irradiance_error:_DeflateLevel = 1 ;

irradiance_error:_Endianness = "little" ;

float nominal_wavelength(xtrack, spectral_channel) ;

nominal_wavelength:units = "nm" ;

nominal_wavelength:valid_min = -9.96921e+36f ;

nominal_wavelength:valid_max = 9.96921e+36f ;

nominal_wavelength:_FillValue = 9.96921e+36f ;

nominal_wavelength:_Storage = "contiguous" ;

nominal_wavelength:_Endianness = "little" ;

ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;

pixel_quality_flag:long_name = "pixel quality flag" ;

pixel_quality_flag:_FillValue = 65535US ;

pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;

pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;

pixel_quality_flag:_Storage = "chunked" ;

pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;

pixel_quality_flag:_Shuffle = "true" ;

pixel_quality_flag:_DeflateLevel = 1 ;

pixel_quality_flag:_Endianness = "little" ;
// group attributes:
} // group band_290_490_nm

group: band_540_740_nm {

dimensions:

xtrack = 2048 ;

spectral_channel = 1028 ;

variables:

int xtrack(xtrack) ;

xtrack:_Storage = "contiguous" ;

xtrack:_Endianness = "little" ;

float irradiance(mirror_step, xtrack, spectral_channel) ;

irradiance:units = "photons/s/cm^2/nm" ;

irradiance:ancillary_variables = "irradiance_error" ;

irradiance:_FillValue = -9.96921e+36f ;

irradiance:_Storage = "chunked" ;

irradiance:_ChunkSizes = 1, 128, 1028 ;

irradiance:_Shuffle = "true" ;

irradiance:_DeflateLevel = 1 ;

irradiance:_Endianness = "little" ;

float irradiance_error(mirror_step, xtrack, spectral_channel) ;

irradiance_error:units = "photons/s/cm^2/nm" ;

irradiance_error:long_name = "irradiance error" ;

irradiance_error:_FillValue = -9.96921e+36f ;

irradiance_error:number_of_significant_digits = 2 ;

irradiance_error:_Storage = "chunked" ;

irradiance_error:_ChunkSizes = 1, 128, 1028 ;

irradiance_error:_Shuffle = "true" ;

irradiance_error:_DeflateLevel = 1 ;

irradiance_error:_Endianness = "little" ;

float nominal_wavelength(xtrack, spectral_channel) ;

nominal_wavelength:units = "nm" ;

nominal_wavelength:valid_min = -9.96921e+36f ;

nominal_wavelength:valid_max = 9.96921e+36f ;

nominal_wavelength:_FillValue = 9.96921e+36f ;

nominal_wavelength:_Storage = "contiguous" ;

nominal_wavelength:_Endianness = "little" ;

ushort pixel_quality_flag(mirror_step, xtrack, spectral_channel) ;

pixel_quality_flag:long_name = "pixel quality flag" ;

pixel_quality_flag:_FillValue = 65535US ;

pixel_quality_flag:flag_masks = 1US, 2US, 4US, 8US, 16US, 32US, 64US, 128US, 256US, 512US, 1024US, 2048US ;

pixel_quality_flag:flag_meanings = "missing_data bad_pixel processing_error transient_pixel rts_pixel saturated noise_underflow dark_corr_error offset_corr_error smear_corr_error straylight_corr_error nonlinear_range" ;

pixel_quality_flag:_Storage = "chunked" ;

pixel_quality_flag:_ChunkSizes = 1, 128, 1028 ;

pixel_quality_flag:_Shuffle = "true" ;

pixel_quality_flag:_DeflateLevel = 1 ;

pixel_quality_flag:_Endianness = "little" ;
// group attributes:
} // group band_540_740_nm

} // group frames

}

danielfromearth · 2024-03-21T16:25:34Z

Thanks @dopplershift!

Hi @ajelenak, does the error message that @dopplershift encountered in his comment above —

NetCDF: Start+count exceeds dimension bound
Location: file ?; fcn ? line 478

— signify anything further to you about what might be causing the netCDF opening issue?

Sounds to me like the dimension information was perhaps recorded wrong somehow? Though I have no idea right now what next to check.

ajelenak · 2024-03-21T17:36:31Z

Wild guess because dimension bounds are mentioned... The spectral_channel netCDF dimensions in the file are HDF5 datasets without any data stored but declared as 1D arrays of certain size. This is valid HDF5, and valid for netCDF dimensions as far as I understand. Reading data from these HDF5 datasets will give all fill values (zeroes in this case).

I tried this file with a h5dump --enable-error-stack=2 ... command to check at the HDF5 level and there were no errors printed.

danielfromearth · 2024-03-21T17:45:23Z

Thanks @ajelenak.

The Location: file ?; fcn ? line 478 message is cryptic. I wonder if anyone here knows what that refers to — perhaps @DennisHeimbigner — e.g., line 478 in which file? And is there any way to determine which variable or dimension might be driving that error, so we can see whether it is the spectral_channel?

dopplershift · 2024-03-21T17:55:25Z

cc @WardF

WardF · 2024-03-21T17:57:31Z

Thanks, I'll take a look. This is not the first time something like this (with this error message) has come up.

DennisHeimbigner · 2024-03-21T19:44:06Z

After some checking, the error report appears to be coming
from the function print_rows in the file ncdump/vardata.c line 478.

WardF · 2024-07-08T21:57:23Z

I am still poking at this to get specifics, thanks for your patience. I was able to use netCDF v4.6.1 to dump the binary file to .cdl, and then use ncgen to re-generate the netCDF binary file; this new, regenerated file does not produce the same error.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

netCDF4 unable to open netCDF file that opens as HDF and with Panoply #1313

netCDF4 unable to open netCDF file that opens as HDF and with Panoply #1313

danielfromearth commented Mar 15, 2024 •

edited

Loading

DennisHeimbigner commented Mar 15, 2024

danielfromearth commented Mar 15, 2024

jswhit2 commented Mar 19, 2024

dopplershift commented Mar 19, 2024

dopplershift commented Mar 19, 2024 •

edited

Loading

danielfromearth commented Mar 21, 2024

ajelenak commented Mar 21, 2024

danielfromearth commented Mar 21, 2024

dopplershift commented Mar 21, 2024

WardF commented Mar 21, 2024

DennisHeimbigner commented Mar 21, 2024

WardF commented Jul 8, 2024

netCDF4 unable to open netCDF file that opens as HDF and with Panoply #1313

netCDF4 unable to open netCDF file that opens as HDF and with Panoply #1313

Comments

danielfromearth commented Mar 15, 2024 • edited Loading

The version of the software with which you are encountering an issue

Environmental information (i.e. Operating System, compiler info, java version, python version, etc.)

A description of the issue with the steps needed to reproduce it

DennisHeimbigner commented Mar 15, 2024

danielfromearth commented Mar 15, 2024

jswhit2 commented Mar 19, 2024

dopplershift commented Mar 19, 2024

dopplershift commented Mar 19, 2024 • edited Loading

danielfromearth commented Mar 21, 2024

ajelenak commented Mar 21, 2024

danielfromearth commented Mar 21, 2024

dopplershift commented Mar 21, 2024

WardF commented Mar 21, 2024

DennisHeimbigner commented Mar 21, 2024

WardF commented Jul 8, 2024

danielfromearth commented Mar 15, 2024 •

edited

Loading

dopplershift commented Mar 19, 2024 •

edited

Loading