Prerequisites

• Active AWS account
• Permissions to create IAM users and policies
• AWS services with active CloudWatch metrics (RDS, EC2, etc.)

Creating an IAM Role for Kondado

To allow Kondado to access CloudWatch metrics, you need to create an IAM user with specific permissions.

Step 1: Access the IAM Console

1. Log in to the AWS Management Console
2. Navigate to the IAM service (Identity and Access Management)
3. In the sidebar, click on Users
4. Click the Add users button

Step 2: Configure the User

1. Enter the username, for example: kondado-cloudwatch-metrics
2. Under "Select AWS credential type", check Access key - Programmatic access
3. Click Next: Permissions

Step 3: Create Permissions Policy

1. Select Attach existing policies directly
2. Click Create policy (will open in a new tab)
3. Select the JSON tab
4. Paste the following policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "cloudwatch:ListMetrics",
           "cloudwatch:GetMetricStatistics",
           "cloudwatch:GetMetricData",
           "cloudwatch:DescribeAlarms"
         ],
         "Resource": "*"
       },
       {
         "Effect": "Allow",
         "Action": [
           "ec2:DescribeInstances",
           "ec2:DescribeVolumes",
           "rds:DescribeDBInstances",
           "rds:DescribeDBClusters",
           "elasticloadbalancing:DescribeLoadBalancers",
           "elasticloadbalancing:DescribeTargetGroups",
           "lambda:ListFunctions",
           "s3:ListAllMyBuckets",
           "dynamodb:ListTables"
         ],
         "Resource": "*"
       }
     ]
   }

5. Click Next: Tags
6. Add optional tags (e.g., Purpose=Kondado)
7. Click Next: Review
8. Give the policy a name: KondadoCloudWatchMetricsReadOnly
9. Add a description: Allows CloudWatch metrics read access for Kondado
10. Click Create policy

Step 4: Attach Policy to User

1. Return to the user creation tab
2. Click the refresh button next to "Create policy"
3. In the search, type KondadoCloudWatchMetricsReadOnly
4. Check the created policy
5. Click Next: Tags
6. Add optional tags
7. Click Next: Review
8. Review the settings and click Create user

Step 5: Save Credentials

1. On the success screen, you'll see the Access key ID and Secret access key
2. IMPORTANT: This is the only time you'll be able to see the Secret access key
3. Click Download .csv to save the credentials
4. Or manually copy the Access key ID and Secret access key
5. Store these credentials in a secure location

Configuring the Data Source in Kondado

1. In Kondado, go to Data Sources → Add New Source
2. Select AWS CloudWatch Metrics
3. Fill in the fields:
   • Source Name: Descriptive name (e.g., "AWS Production")
   • AWS Access Key ID: Paste the IAM user's Access Key ID
   • AWS Secret Access Key: Paste the IAM user's Secret Access Key
   • AWS Region: Region where your resources are (e.g., us-east-1, sa-east-1)
4. Click Test Connection to verify credentials
5. If the test passes, click Save

Creating a Pipeline

After configuring the data source, you can create pipelines to collect metrics:

1. Go to Pipelines → Create New Pipeline
2. Select the created AWS CloudWatch Metrics data source
3. Choose the pipeline type:
   • Metrics (Incremental): Collects new data since last execution
   • Metrics (Rolling Window): Always collects the last X days (e.g., last 7 days)
4. Configure the parameters:
   • Namespace: AWS service (e.g., AWS/RDS, AWS/EC2)
   • Metric: Metric name (e.g., CPUUtilization, ReadLatency)
   • Dimension: Optional filter (e.g., DBInstanceIdentifier)
   • Dimension Value: Specific value or leave empty for all instances
   • Statistics: Select one or more (Average, Sum, Minimum, Maximum, SampleCount)
   • Period: Granularity in minutes (e.g., 5 minutes)
5. Configure the destination (BigQuery, PostgreSQL, etc.)
6. Schedule the pipeline execution
7. Click Create Pipeline

Configuration Examples

Example 1: CPU of all RDS instances

• Namespace: AWS/RDS
• Metric: CPUUtilization
• Dimension: DBInstanceIdentifier
• Dimension Value: (leave empty for all)
• Statistics: Average, Maximum
• Period: 5 minutes

Example 2: Read latency of a specific database

• Namespace: AWS/RDS
• Metric: ReadLatency
• Dimension: DBInstanceIdentifier
• Dimension Value: database-name
• Statistics: Average
• Period: 1 minute

Important Notes

• If you leave "Dimension Value" empty, Kondado will automatically discover all instances and collect data from each one
• The dimension column (e.g., dbinstanceidentifier) will appear in the destination table with each instance's name
• For incremental pipelines, set an initial Savepoint (date from which to start collecting)
• For rolling window pipelines, configure the window size in days
• You can select multiple statistics - each will appear as a separate column

Troubleshooting

Error: "Invalid AWS credentials"

• Verify that the Access Key ID and Secret Access Key are correct
• Confirm that the IAM user was not disabled or deleted

Error: "Access denied"

• Verify that the IAM policy was correctly attached to the user
• Confirm that the specified region is correct

No metrics returned

• Verify that the specified AWS service has active metrics
• Confirm that the namespace is correct (e.g., AWS/RDS, not rds)
• For dimension-specific metrics, verify that the dimension value exists

• Metrics (Incremental)
• Metrics (Rolling Window)