Firehose Setup

MetaRouter Firehose Setup enables seamless event data streaming to cloud-based destinations like Amazon Kinesis, Google Pub/Sub, and Apache Kafka. This guide walks you through configuring Firehose to efficiently route data to your preferred platform for real-time analytics and processing.

Set up Firehose

Create User
1. Open the Identity and Access Management (IAM) console.
2. In the sidebar, go to Users.
3. Click Create User.
4. Enter a username, then click Next.
5. Configure permissions:
  1. Select Attach policies directly.
  2. Attach the following policies:
    - AmazonKinesisFirehoseFullAccess
    - AmazonS3FullAccess
    - AmazonRedshiftAllCommandsFullAccess
    - AmazonRedshiftDataFullAccess
    - AmazonRedshiftFullAccess
    - CloudWatchLogsFullAccess
  3. Click Create User.
Get Access Keys
1. Click the user you just created.
2. Click Create Access Key.
3. Select a use case for the key.
4. (Optional) Enter a description.
5. Click Create Access Key.
6. Note down your Access Key and Secret Key, as the Secret Key is only visible once.
Create a Role and Edit Trust Policy
1. In the IAM console, go to Roles from the sidebar.
2. Click Create Role.
3. Under Trusted Entity Type, select AWS service (default).
4. Under Use case, select Redshift.
5. Keep Redshift - Customizable as the default use case, then click Next.
6. Attach the following permissions:
  - AmazonKinesisFirehoseFullAccess
  - AmazonS3FullAccess
  - AmazonRedshiftAllCommandsFullAccess
  - AmazonRedshiftDataFullAccess
  - AmazonRedshiftFullAccess
  - CloudWatchLogsFullAccess
7. Click Next.
8. Enter the role details:
  - Role name
  - (Optional) Role description
9. Click Create Role.
10. Open the newly created role and go to the Trust Relationships tab.
11. Click Edit Trust Policy.
12. Replace the existing policy with the following JSON:
```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "firehose.amazonaws.com",
                    "redshift.amazonaws.com",
                    "s3.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
```

Create an S3 bucket
1. Open the Amazon S3 console.
2. Click Create Bucket.
3. Enter a unique bucket name.
4. Keep the default settings, or adjust them based on your requirements.
5. Click Create Bucket.