Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

effect of max_seq_length on performance #50

Open
Ashbajawed opened this issue Oct 19, 2020 · 0 comments
Open

effect of max_seq_length on performance #50

Ashbajawed opened this issue Oct 19, 2020 · 0 comments

Comments

@Ashbajawed
Copy link

I was getting memory allocation error while fine tuning mega model I then reduce batch size to 1 and training is now on going. I also tried to reduce max_seq_length to 512 and set batch_size to 4 and it was working.

My questions is what parameter will effect more on performance reducing batch size or reducing max_seq_length ?

Also can I set the value of max_seq_length other then the power of 2 like some value between 512 and 1024?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant