Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Models: documented public methods/attributes to extract best parameters for Auto* models #829

Open
yarnabrina opened this issue Apr 28, 2024 · 3 comments

Comments

@yarnabrina
Copy link

Description

Especially for the auto models it is quite common for users to know which parameter combination is the best model for reporting/analyses purposes, and also to know what are the corresponding AIC/BIC/etc. values. So, something along the lines of the following will be useful:

  1. get_best_parameters() or best_paramaters
  2. get_best_model_score or best_model_score

Use case

This is a request raised multiple times by sktime users for the _GeneralisedStatsForecastAdapter adapter. Using this adapter, we have exposed all the 5 auto models: AutoARIMA, AutoTheta, AutoETS, AutoCES, AutoTBATS, and a few non-auto models.

As of now, model_ attribute gives few details, but the keys seem to vary for different algorithms and it is difficult to expose parameters/scores reliably for all models using this.

@yarnabrina
Copy link
Author

@AzulGarza @jmoralez , can you please comment on this issue? Is it safe to add a get_fitted_params methon in statsforecast wrapper of sktime relying on model_ attribute, or will you recommend something else?

The same question also applies for neuralforecast probably.

@jmoralez
Copy link
Member

jmoralez commented May 9, 2024

Hey @yarnabrina, sorry for the late reply. I think we can add it on our side, it would probably also fix #774. Just to be sure, these would return the values that are tuned, right? For example for AutoETS it would return model and damped.

@yarnabrina
Copy link
Author

Just to be sure, these would return the values that are tuned, right? For example for AutoETS it would return model and damped.

Yes, right. That'll help users to specify those themselves in future in case they want to avoid the tuning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants