Step 1: Setup GitHub REST API Requests 🤙

[nelsonic]

In order to get data from GitHub we need to setup the API.

According to the docs: https://docs.github.com/en/rest/overview/resources-in-the-rest-api 👀
the rate limiting for authenticated requests is 5,000/hour. 5️⃣ 🦘
That should be more than "enough" for what I have in mind. 📈
But if we need more we can easily add more API users. 💭

We've made HTTP Requests to the GitHub API before in previous projects:

• https://github.com/dwyl/elixir-auth-github
• https://github.com/dwyl/github-backup and
• https://github.com/nelsonic/issues

So I think I can piece this together from previous code. 🤞

Todo

• Setup GitHub API access and make a test request
• Use our existing knowledge/tools to fetch a list of @dwyl Org Members from the GitHub API

[nelsonic]

Decided to use https://github.com/edgurgel/tentacat for the GitHub API requests as it appears to be maintained. 👍

[nelsonic]

Used tentacat for basic requests and they worked ... 🎉
but then I re-read the GitHub REST API Docs:
https://docs.github.com/en/rest/orgs/members#list-organization-members

The GET /orgs/{org}/members request defaults to returning 30 members ...

@dwyl currently has 525 members: https://github.com/orgs/dwyl/people

We definitely need to paginate the request:

if we're unable to paginate the results this approach is a non-starter. 🤷‍♂️

Tentacat.Organizations.Members.list/2 function signature has only client and organisation arguments:
https://github.com/edgurgel/tentacat/blob/dfe89f33350acdf1ddea4a6f4d4338932fca0875/lib/tentacat/organizations/members.ex#L64-L67

 @doc """ 
 List members of a `organization`. 

 The response will differ if the authenticated user is also owner of the 
 organization. 

 ## Example 

     Tentacat.Organizations.Members.list "github" 
     Tentacat.Organizations.Members.list client, "github" 

 More info at: http://developer.github.com/v3/orgs/members/#members-list 
 """ 
 @spec list(Client.t(), binary) :: Tentacat.response() 
 def list(client \\ %Client{}, organization) do 
   get("orgs/#{organization}/members", client) 
 end 

Opened issue to confirm my understanding: edgurgel/tentacat#210 🤞

[nelsonic]

Might be able to use Tentacat.get/3 generic HTTP function to manually get the endpoint.

https://github.com/edgurgel/tentacat/blob/dfe89f33350acdf1ddea4a6f4d4338932fca0875/lib/tentacat.ex#L55-L95

@doc """
Underlying utility retrieval function. The options passed affect both the
return value and, ultimately, the number of requests made to GitHub.

## Options

  * `:pagination` - Can be `:none`, `:manual`, `:stream`, or `:auto`. Defaults to :auto.

      - `:none` will only return the first page. You won't have access to the
        headers to manually paginate.

      - `:auto` will block until all the pages have been retrieved and
        concatenated together. Most of the time, this is what you want. For
        example, `Tentacat.Repositories.list_users("chrismccord")` and
        `Tentacat.Repositories.list_users("octocat")` have the same interface
        though one call will page many times and the other not at all.

      - `:stream` will return a `Stream`, prepopulated with the first page.

      - `:manual` will return a 3 element tuple of `{page_body,
        url_for_next_page, auth_credentials}`, which will allow you to control
        the paging yourself.
"""
@spec get(binary, Client.t()) :: response
@spec get(binary, Client.t(), keyword) :: response
@spec get(binary, Client.t(), keyword, keyword) ::
        response | Enumerable.t() | pagination_response
def get(path, client, params \\ [], options \\ []) do
  url =
    client
    |> url(path)
    |> add_params_to_url(params)


  case pagination(options) do
    nil -> request_stream(:get, url, client.auth)
    :none -> request_stream(:get, url, client.auth, "", :one_page)
    :auto -> request_stream(:get, url, client.auth)
    :stream -> request_stream(:get, url, client.auth, "", :stream)
    :manual -> request_with_pagination(:get, url, client.auth)
  end
end

Thanks to edgurgel/tentacat#190 (comment) (comment on super-stale PR) 👀

Going to take a look at this on the Weekend. 🤞
Need to park this for now as it's lower priority than building the App/MVP. 😞

[nelsonic]

Going to try and cURL my way into figuring out the pagination ...

curl \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <YOUR-TOKEN>" \
  https://api.github.com/orgs/dwyl/members

Again, one for Saturday. ⏳