Github APIs provide a wide variety of data, such as events, repos, and users. I implemented a few charts for one authenticated user, specifically, to see your popular repos by fork_count, stargazers, watchers etc. The thing is, in spite of these per-repo-based activity data, it is not easy to get aggregated data for repos, it is either beyond the request limit or there is no historical data(for example, you can retrieve activity data within 90 days). After looking around on the Internet, the only feasible way seems to be using Github Archive Data, which is stored and updated in Google BigQuery.
You can activate Google BigQuery as mentioned here, once you get access to this dataset, you can query data similar to the way you use SQL(but not exactly the same), the you can download the result CSV file.
However, in my case, I need to query relevant data for each user, and hopefully update query within latest month dataset. CSV files won’t work then.
Fortunately, google provides APIs including BigQuery api. I should be able to get data similar to the way I query Github API.(I also kept the Github API in Express Router as well).
So I’d like to take notes of both libraries, hopefully I can unify the APIs either use Python or a stable Node.js client library.
The official documentation is a good start. And just a reminder, we need to use OAuth2 for google apis as well.
Basically, what you need to do is to use the client.js to initialize the process, and authorize the request with some essential information such as
project_id, and ‘scope’.
Node.js Client Library
The node.js client library OAuth2 is similar to what was mentioned in previous posts. Users click on auth button, they will be redirected to a
authorization_url which is generated by the library, the
authorization_url is a consent page where users confirm the authentication, after that, page will be redirected to the callback url you specified in your app, with a
code parameter. The next step is to use this code to exchange an token, save that token and use this token for each request.
- BigQuery Big Data Visualization With D3.js;
- Google API Node JS Authentication and Usage;