Setting up the BigQuery Environment¶
The BigQuery connector allows you to access the BigQuery REST API through WSO2 EI.
To work with the BigQuery connector, you need to have a Google Cloud Platform account. If you do not have a Google Cloud Platform account, go to console.cloud.google.com, and create a Google Cloud Platform trial account.
BigQuery uses the OAuth 2.0 protocol for authorization. All requests to the BigQuery REST API will be authorized against a registered user. Developers can generate user credentials from the Google Cloud Platform using two different mechanisms. See the following sections for detail.
Obtaining user credentials¶
Follow the steps below to generate user credentials.
Obtaining a client ID and client secret
-
Go to https://accounts.google.com/SignUphttps://accounts.google.com/SignUp and create a Google account.
-
Go to https://console.developers.google.com/projectselector/apis/credentials, and sign in to your Google account.
-
If you do not already have a project, you can create a new project. Click Create credentials and then select OAuth client ID.
-
Next, select Web Application, and create a client.
-
Add https://www.google.com as the redirect URI (you can add any URI that you need as redirect URI) under Authorized redirect URIs, and then click Create. This displays the client ID and client secret.
-
Make a note of the client ID and client secret that is displayed, and then click OK.
-
Click Library on the left navigation pane.
-
Search BigQuery API under the Big data category.
-
Click Enable. This enables the BigQuery API.
-
Get the authorization code by sending a GET request to the following URL. Replace the
<redirect_uri>
and<client_ID>
with the redirect uri and client ID values noted in the previous steps. Enter the following URL in your web browser.
Note the authorization code for future use.https://accounts.google.com/o/oauth2/auth?redirect_uri=<redirect_uri>&response_type=code&client_id=<client_ID>&scope=https://www.googleapis.com/auth/bigquery&approval_prompt=force&access_type=offline
-
Get the
access token
andrefresh token
by sending a POST request to the URL given below. Be sure to use an x-www-form-urlencoded body with the<authorization_code>
,<client_id>
,<client_secret>
, and<redirect_uri>
values noted before, and also set thegrant_type
to authorization_code. You will need them to configure WSO2 EI Big Query Connector.https://www.googleapis.com/oauth2/v3/token.
Obtaining credentials using the service account¶
-
Open the Service Accounts page in the GCP console.
-
Select your project and click Open.
-
Click Create Service Account.
-
Enter Service account details, and then click Create.
-
Select a role you wish to grant to the service account and click Continue.
-
Grant users access to this service account (optional) and click Done.
-
Go to the service account for which you wish to create a key and click the created Service account in that row.
-
Click Create key.
-
Select the key type as P12 and click Create. Then the created key will be downloaded.
Creating Project, Dataset and Table¶
Creating Project
-
Open the BigQuery console.
-
Click down arrow key shown in the following image.
-
Click New Project.
-
Enter new project details.
Creating Dataset
-
After creating the Project, click the created project. You can see the following details. Then click Create Dataset.
-
Enter required Dataset details and click Create Dataset.
Creating Table
-
After creating the Dataset, click the created Dataset. You can see the following details. Then click Create Table.
-
Enter required Table details and click Create.
For more information about these operations, please refer to the BigQuery connector reference guide.
Top