As explained in the presentation article, Stambia can work with Google Cloud Platform to perform operations on technologies such as Google BigQuery or Google Cloud Storage.
To accomplish this, you'll first have to install the Stambia's Google Cloud Platform Connector and prepare your environment.
You can find below all the explanations and instructions to install and set up everything to get ready to work with Google Cloud Platform (GCP).
This article explains moreover how to create and configure Google Cloud Platform Metadata.
Prerequisites:
- Stambia DI Designer S18.3.8 or higher
- Stambia DI Runtime S17.4.7 or higher
- Java 1.8
Installation
Introduction
The installation is composed of the following steps:
- Download
- Download of the Connector
- Download of the Templates
- Installation
- Installation of the Connector
- Installation of the Templates
- Additional steps
Download
The first step is to download all the necessary materials that you'll need to perform the installation.
Lead to the presentation article and retrieve the following items from the download section.
Google Cloud Platform Connector
This archive contains the connectors to install in Stambia DI Designer to be able to work with Google Cloud Platform.
This includes all the necessary for Google Cloud Platform, Google Cloud Storage, and Google BigQuery.
Templates
The Templates will be used to perform Mappings and Processes in Stambia DI Designer.
You have to download Generic Templates, Google Cloud Platform Templates, and Google BigQuery Templates
Connector Installation
Once you've downloaded the Google Cloud Platform Connector, proceed to the installation of the plugin as usual.
You can follow this link to find how to install Connectors in the Designer.
The Google Cloud Platform Connector is no different than any other additional Connector.
The only difference is that it contains two addons, one for Google Cloud Platform / Google Cloud Storage, and one for Google BigQuery.
When the installation is finished, you can start the Designer and proceed the next steps.
Templates Installation
The next step consists of importing all Templates archives downloaded previously.
Start your Designer and import them as usual in your workspace.
Remember that you have to import three pack of Templates: Generic, Google Cloud Platform, and Google BigQuery.
Additional Steps
JDBC Drivers Configuration
One additional step is required if you plan to work with Google BigQuery.
You'll have to configure the Google BigQuery entry in JDBC Drivers Preferences.
It is necessary to configure it if you want to connect, reverse or consult data through the Designer on Google BigQuery.
The entry in the Designer is already pre-configured and all you have to do is to modify it to add the additional libraries.
Open the Window Menu > Preferences > SQL Explorer > JDBC Drivers > select "Google BigQuery" and click on Edit
Then, lead to the "Extra Class Path" tab and click on "Add JARs".
Use the Popup to select all the Google BigQuery libraries which have been automatically extracted by the connector to <Designer Installation Directory>/stambiaRuntime/lib/addons/bigquery
You can use shortcut as "CTRL+a" to select everything if required.
Then, click on OK to save your modifications.
The screenshot only shows some of the additional libraries as an example, you must add here all the libraries
If the Google BigQuery entry does not exist in JDBC Drivers, you can add it manually. The needed information will be the following:
Example URL: jdbc:stambia:bigquery / Driver Class Name: com.stambia.jdbc.bigquery.BigQueryDriver (retrieved from stambia.udriver.bigquery.jar)
Metadata Definition
When Stambia DI Google Cloud Platform Connector is installed and configured, you can start creating your firstGoogle Cloud Platform Metadata.
This Metadata is the main Metadata that will be used to centralize the credentials that will be used in other Metadata such as Google Storage.
It represents a Google Cloud Project and the information to connects to this project.
Metadata creation
Create the Metadata, as usual, by selecting the technology in the Metadata Creation Wizard:
Click next, choose a name and click on finish.
Metadata configuration
First information to define in Metadata is the Google Cloud Project Identifier, which is the unique identifier of the project you want to work on.
Check with your administrator or lead to Google Cloud Platform's console page in your browser to find it, and then define it in Metadata.
Next step is to define credentials to connect and perform operations on the project.
For this, create a credential node under the root node and define one of the two "JSon Key File URI" or "JSon Key String" depending on your preference.
In both cases, the credential represent the JSON private key file containing the access token, which can be generate from Google Cloud Platform's by administrators.
Here is some information about these two attributes:
| Name |
Description |
| JSon Key File URI |
Absolute file path to the JSON private key file corresponding to the account to connect with. Note that it must be accessible by the Runtime on the file system. |
| JSon Key String |
JSon Access token string. This corresponds to the content of the JSON private key file corresponding to the account to connect with. The advantage is that, instead of using a file path as with the other attribute, you specify the credentials directly in Metadata, so you do not have to store a local file somewhere that can be accessed by the Runtime. To fill this attribute, open your JSON private key file, copy its content and paste it in the attribute. Note that you must remove the line returns contained in this file: this attribute is expected a single line JSon credential string. |
Only one of the two attributes should be defined.
Here is an example of JSON credential file:
{
"type": "service_account",
"project_id": "mygoogleproject",
"private_key_id": "myprivatekey",
"private_key": "myprivatekeyvalue",
"client_email": "myaccountemail",
"client_id": "myclientid",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "mycerturl"
}
Demonstration Project
Demonstration project presenting common and advanced usage of the Google Cloud Platform in Stambia DI can be found on the download page.
You can download and import it in your workspace and then have a look at the Mappings and Processes examples.
It is a good start to familiarize with its usage and to see how the Metadata are configured, for instance.