'Listing GitHub users with their repos count

I want to list GitHub users with their repos count using GitHub rest API.

I tried to send request (https://api.github.com/search/users?q=${q}&&per_page=${COUNTS_PER_PAGE}&&page=${page || 1}) using axios in simple node app. But there is no repo-related field in response except repos_url.

So I tried to send request with repos_url again per users but met rate limit exceeded.

Is there anybody help me with solving this issue? Any help will be helpful for me.

node.js


Solution 1:[1]

I ran into this same issue with scraping one of the Organizations I'm involved in. The way we got around it was to use python and do a try/except block to keep trying over and over.

while True:
    params = {"cursor": endCursor}
    try:
        response = get_repos(params)
    except Exception:
        rand_sleep = randint(5, 30)
        print(f"Gonna sleep for {rand_sleep} because of a 403 error...")
        sleep(rand_sleep)
    repos = response['organization']['repositories']['nodes']
    all_repos = all_repos + repos
    hasNextPage = response['organization']['repositories']['pageInfo']['hasNextPage']
    endCursor = response['organization']['repositories']['pageInfo']['endCursor']

We leverage the https://api.github.com/graphql endpoint to get the data we need, and then just keep trying over and over to get the chunks of data.

Solution 2:[2]

The repo count can be retrieved using the superface sdk instead of calling the api using axios manually. the sdk handles the communication with the github API and returns the repo list , then you can get number from there. here are the steps

npm install @superfaceai/one-sdk

npx @superfaceai/cli install vcs/user-repos


const { SuperfaceClient } = require('@superfaceai/one-sdk');

const sdk = new SuperfaceClient();

async function run() {
  // Load the installed profile
  const profile = await sdk.getProfile('vcs/user-repos');

  // Use the profile
  const result = await profile
    .getUseCase('UserRepos')
    .perform({
      user: 'superfaceai'
    });

  console.log(result.unwrap().repos.length);
}

run();

You can make the script to sleep in between requests to cope with the rate limiting

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 JJ Asghar
Solution 2

Related Questions