Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add recommendations on --n-robust-runs and --maxit #1148

Open
tsalo opened this issue Nov 16, 2024 · 3 comments
Open

Add recommendations on --n-robust-runs and --maxit #1148

tsalo opened this issue Nov 16, 2024 · 3 comments
Labels
documentation issues related to improving documentation for the project

Comments

@tsalo
Copy link
Member

tsalo commented Nov 16, 2024

Summary

We have some info on robustica in the FAQ, but it's not clear what folks should use for --n-robust-runs (and --maxit I suppose). My big question is whether selecting the number of robust runs is just a matter of resources and time (i.e., should you use the highest number you're willing to wait for) or if there's a sweet spot (i.e., increasing the number runs will result in fewer consistent components, which means that more is not necessarily better).

If we don't actually know the answer to this, I could try running tedana on a single run with different --n-robust-runs values to see how the parameter impacts the number of detected components.

@tsalo tsalo added the documentation issues related to improving documentation for the project label Nov 16, 2024
@eurunuela
Copy link
Collaborator

I would say you can expect the robust number of components to stay consistent across an infinite number of runs. I don't know how long it takes to run tedana with robustica, but 5-10 runs should be enough in my opinion.

@handwerkerd
Copy link
Member

@tsalo and I just discussed this at the end of our developers' call. It would be useful to run a bunch of options on a few datasets to test/show that are defaults are reasonable. This would be better to do after we add the index quality for each component and the TSNE quality results to our outputs so that we have data to show using tools that others can also run themselves.

As for --n-robust-runs, my understanding is that @BahmanTahayori tried various options and found that 30 gave stable results such that the final components were very similar if robustICA was rerun with a different seed and that more iterations didn't noticable improve the quality of the results.

@BahmanTahayori
Copy link
Contributor

I ran robustICA on 20 subjects from our dataset with --n_robust_runs set to [10 20 30 ... 100] For most subjects, the detected active volume plateaued at 20 runs, though a few required more (30). Thus, I set the default number of runs to 30. However, depending on the quality of the dataset, additional runs may be required. I will work on adding TSNE plots to the code which should help the user make more informed decisions about the number of runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation issues related to improving documentation for the project
Projects
None yet
Development

No branches or pull requests

4 participants