FTP

FTP is a protocol for transferring files over the internet. The FTP data pipeline for the Data Warehouse created by Kondado allows you to have access to the data of your files in your analytical cloud.

Adding the data source

To automate FTP ETL with Kondado for your database or spreadsheet, follow the steps below:

1) Have your FTP service address, port, username and password handy

2) Allow the Kondado IPs on your FTP server

3) On the Kondado platform, go to add data sources page and select the FTP data source

4) Name your data source and enter the information from step (1)

When informing the “Address” parameter, use only the host, as shown in the image – without including “ftp://” or even “/” at the end

Now just save the connector and start integrating your FTP data into the Data Lake or Data Warehouse.

Pipelines

Relationship Chart

CSV

You can indicate the name of a file or even the beginning of the file name and we will integrate all of them.

Once executed, the pipeline will save the highest change date of the files it read and, on the next run, only look for files that have a later change date.

In order to absorb files with different columns, the data will be pivoted on the target and will follow the following pattern:

Field	Type
row_number	int
column_number	int
first_column_value	text
value	text
__file_basename	text
__file_path	text
__file_name	text
__kdd_insert_time	timestamp