Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy demo job is failing due to rate limit #79

Open
simonw opened this issue Feb 3, 2023 · 2 comments
Open

Deploy demo job is failing due to rate limit #79

simonw opened this issue Feb 3, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@simonw
Copy link
Collaborator

simonw commented Feb 3, 2023

https://github.com/dogsheep/github-to-sqlite/actions/runs/4080058087/jobs/7032116511

@simonw simonw changed the title Deploy demo job is failing Deploy demo job is failing due to rate limit Apr 5, 2023
@simonw simonw added the bug Something isn't working label Apr 5, 2023
@simonw
Copy link
Collaborator Author

simonw commented Apr 5, 2023

It's a rate limiting problem: https://github.com/dogsheep/github-to-sqlite/actions/runs/4622674009/jobs/8175633155

  File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/cli.py", line 407, in commits
    utils.save_commits(db, commits, repo_full["id"])
  File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 635, in save_commits
    for commit in commits:
  File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 417, in fetch_commits
    for commits in paginate(url, headers):
  File "/home/runner/work/github-to-sqlite/github-to-sqlite/github_to_sqlite/utils.py", line 478, in paginate
    raise GitHubError.from_response(response)
github_to_sqlite.utils.GitHubError: ('API rate limit exceeded for user ID 9599.', 403)
Error: Process completed with exit code 1.

simonw added a commit that referenced this issue Apr 5, 2023
@nedbat
Copy link
Contributor

nedbat commented Dec 8, 2023

Adding &per_page=100 would reduce the number of API requests by 3x.

nedbat added a commit to nedbat/github-to-sqlite that referenced this issue Dec 9, 2023
simonw pushed a commit that referenced this issue Dec 10, 2023
* always ask for 100 items when paginating (helps #79)
* fix typos in README.md
* ignore test and build artifacts
* --org and --state options for pull-requests
* --search for pull-requests, but it can only get 1000
simonw added a commit that referenced this issue Dec 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants