Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ValueError: unable to infer dtype on variable 'time'; xarray cannot serialize arbitrary Python objects #9194

Closed
3 of 12 tasks
berland opened this issue Nov 12, 2024 · 2 comments
Labels

Comments

@berland
Copy link
Contributor

berland commented Nov 12, 2024

What happened? (You can include a screenshot if it helps explain)

This error is observed in the logs:

...
File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 309, in encode_cf_variable
    var = ensure_dtype_not_object(var, name=name)
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 265, in ensure_dtype_not_object
    data = _copy_with_dtype(data, dtype=_infer_dtype(data, name))
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 198, in _infer_dtype
    raise ValueError(
ValueError: unable to infer dtype on variable 'time'; xarray cannot serialize arbitrary Python objects

#6525 has probably reoccured.

What did you expect to happen?

No crash. We should support dates beyond 2262 if that is the problem, and assert it in tests.

steps to reproduce

Not known.

Environment where bug has been observed

  • python 3.8
  • python 3.9
  • python 3.10
  • python 3.11
  • python 3.12
  • macosx
  • rhel7
  • rhel8
  • local queue
  • lsf queue
  • slurm queue
  • openPBS queue
@berland
Copy link
Contributor Author

berland commented Nov 12, 2024

Full traceback:

Traceback (most recent call last):
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/ert/callbacks.py", line 69, in _write_responses_to_storage
    run_arg.ensemble_storage.save_response(
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/ert/storage/mode.py", line 105, in inner
    return func(self_, *args, **kwargs)
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/ert/storage/local_ensemble.py", line 847, in save_response
    data.to_netcdf(output_path / f"{response_type}.nc", engine="scipy")
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/core/dataset.py", line 1912, in to_netcdf
    return to_netcdf(  # type: ignore  # mypy cannot resolve the overloads:(
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/backends/api.py", line 1232, in to_netcdf
    dump_to_store(
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/backends/api.py", line 1279, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/backends/common.py", line 266, in store
    variables, attributes = self.encode(variables, attributes)
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/backends/common.py", line 355, in encode
    variables, attributes = cf_encoder(variables, attributes)
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 899, in cf_encoder
    new_vars = {k: encode_cf_variable(v, name=k) for k, v in variables.items()}
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 899, in <dictcomp>
    new_vars = {k: encode_cf_variable(v, name=k) for k, v in variables.items()}
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 309, in encode_cf_variable
    var = ensure_dtype_not_object(var, name=name)
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 265, in ensure_dtype_not_object
    data = _copy_with_dtype(data, dtype=_infer_dtype(data, name))
  File "/prog/res/komodo/2024.10.07-py38-rhel8/root/lib64/python3.8/site-packages/xarray/conventions.py", line 198, in _infer_dtype
    raise ValueError(
ValueError: unable to infer dtype on variable 'time'; xarray cannot serialize arbitrary Python objects


@berland berland moved this to Todo in SCOUT Nov 13, 2024
@oyvindeide
Copy link
Collaborator

This was fixed in: #8684, and is available in ert >= 11.1

@github-project-automation github-project-automation bot moved this from Todo to Done in SCOUT Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

No branches or pull requests

2 participants