SQL Server SSH data integration for Data Warehouse created by Kondado enables you to have access to SQL Server SSH data in your analytics cloud.
Adding the data source
To automate SQL Server SSH ETL with Kondado for your database, follow the steps below:
2) On the database server, allow the connection to the intermediary server on the port that will be used
4) Give your data source a name and fill in the information for your intermediate server and then your database. In “Server PEM key” just copy the text of the file with the PEM key.
Now just save the connector and start replicating your SQL Server SSH data into the Data Lake or Data Warehouse.
Pipelines
Tables and Views
- With our pipelines, you will be able to replicate tables as well as views
- If your table has a datetime/timestamp type column that marks when a record was changed/inserted, your pipeline can be incremental
- It will be necessary to define the primary key, which can be defined by several columns and refers to the column (or set of columns) that define a record as being unique
The table created in the destination will have a format similar to the one below:
Add Microsoft SQL Server SSH as a Data Source on Kondado
Configure a secure SSH tunnel connection to your Microsoft SQL Server database through an intermediary bastion server on the Kondado platform.
Allow Kondado IPs on your database server
Configure your database server and VPC firewall to allow incoming connections from Kondado IPs on the port used for SQL Server. This is the first security layer for your secure data integration.
Enable connection to the intermediary server
On the database server, allow outbound connections to your SSH bastion/tunnel server on the port that will be used for the tunnel.
Select the Microsoft SQL Server SSH connector
Log in to the Kondado platform, navigate to add data sources, and choose the Microsoft SQL Server SSH data source from the available connectors.
Configure the SSH tunnel and database credentials
Give your data source a name, then fill in the intermediate server (bastion) details followed by your actual database connection info. For "Server PEM key", paste the full text content of your PEM key file.
Save and start replicating data
Save the connector configuration. You can now begin replicating your SQL Server SSH data into your Data Lake or Data Warehouse, with support for both tables and views.
Set up incremental replication if applicable
For incremental pipelines, ensure your table has a datetime/timestamp column tracking record changes/insertions, and define the primary key (which may span multiple columns) to uniquely identify records.