Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Let TensorFlow accessor and data loader handle either xarray.DataArray or xarray.Dataset inputs #107

Merged
merged 7 commits into from
Oct 13, 2022

Conversation

norlandrhagen
Copy link
Contributor

Description of proposed changes

Adds to: #83 (comment). Based off changes in PR: #85.

  • method _as_xarray_dataarray moved outside of TorchAccessor class.
  • _as_xarray_dataarray method modified to use is_instance instead of try:except.
  • unit test added to cover _as_xarray_dataarray
  • Attempt at KerasAccessor and corresponding tests.

@codecov-commenter
Copy link

codecov-commenter commented Oct 12, 2022

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (ce6fbfd) to head (e4f6ef1).
Report is 76 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #107   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            5         5           
  Lines          182       190    +8     
  Branches        33        35    +2     
=========================================
+ Hits           182       190    +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines 44 to 45
@xr.register_dataarray_accessor("keras")
@xr.register_dataset_accessor("keras")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure that we want to use keras instead of tensorflow as the accessor name (considering that the returned tensors are tf.Tensor objects)?

Copy link
Member

@weiji14 weiji14 Oct 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, I guess we're using xbatcher.loaders.keras.CustomTFDataset in the dataloader at https://github.com/xarray-contrib/xbatcher/blob/ed45a99da54503de2e94cc90f12510f590ea9be6/doc/api.rst#dataloaders, so might as well stick with keras to be consistent (unless anyone is keen to change everything to tensorflow).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your point. What do you and @norlandrhagen think of tf for the accessor name and TFAccessor as the class name to keep it shorter?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedbaack @weiji14 @maxrjones

Good point on the naming. I can update the accessor name and class name. Should up update the data loader naming(xbatcher.loaders.keras.CustomTFDataset) in this PR?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rename the tensorflow dataloader in a separate PR (so that it shows up in the changelog as a backward incompatible change).

xbatcher/accessors.py Outdated Show resolved Hide resolved
Comment on lines 44 to 45
@xr.register_dataarray_accessor("keras")
@xr.register_dataset_accessor("keras")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your point. What do you and @norlandrhagen think of tf for the accessor name and TFAccessor as the class name to keep it shorter?

@weiji14 weiji14 changed the title KerasAccessor and _as_xarray_dataarray update TFAccessor and _as_xarray_dataarray update Oct 13, 2022
xbatcher/accessors.py Outdated Show resolved Hide resolved
xbatcher/tests/test_accessors.py Outdated Show resolved Hide resolved
@norlandrhagen norlandrhagen merged commit cb2cd85 into xarray-contrib:main Oct 13, 2022
@maxrjones maxrjones mentioned this pull request Oct 17, 2022
5 tasks
@maxrjones maxrjones changed the title TFAccessor and _as_xarray_dataarray update Let TensorFlow accessor and data loader handle either xarray.DataArray or xarray.Dataset inputs Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants