MySQL is an open-source relational database that powers many applications and is one of the most popular databases in the world. The MySQL data integration for the Data Warehouse created by Kondado allows you to replicate your tables and views to your analytical cloud, integrally or incrementally (capturing only new and updated data).
Adding the data source
To automate MySQL ETL via SSH with Kondado for your database, follow the steps below:
2) On the database server, release the connection to the intermediary server on the port that will be used (usually 3306).
4) Give your data source a name and fill in the information for your intermediate server and then your database. In “Server PEM key” just copy the text of the file with the PEM key.
Now just save the data source and start integrating MySQL data into the Data Lake or Data Warehouse via SSH.
Pipelines
Relationship Chart
Tables and Views
With our pipeline, you will be able to integrate tables as well as views
If your table has a datetime/timestamp type column that marks when a record was changed/inserted, your pipeline can be incremental
It will be necessary to define the primary key, which can be defined by several columns and refers to the column (or set of columns) that define a record as being unique
The created table will have a format similar to the one below:
Add MySQL SSH data source on Kondado
Connect to a MySQL database through an SSH tunnel (bastion server) on the Kondado platform for secure ETL replication.
Whitelist Kondado IPs on the intermediary server
Add the Kondado IPs to your intermediary server (and VPC) for the SSH port that will be used, usually port 22. This is the first step to establish a secure tunnel for your data integration pipeline.
Allow database server connection from the intermediary
On your actual database server, release the connection to the intermediary server on the database port, typically 3306 for MySQL. This ensures the bastion can reach your database.
Select MySQL SSH data source on Kondado
On the Kondado platform, navigate to the add data sources page and select the MySQL data source option to begin configuring your SSH connection.
Configure server and database credentials
Give your data source a name and fill in the information for your intermediate server first, then your database details. In the "Server PEM key" field, paste the full text content of your PEM key file.
Save and start integrating
Save the data source and begin integrating MySQL data into your Data Lake or Data Warehouse via SSH. You can now set up pipelines for tables and views, including incremental replication if you have a datetime/timestamp column.