Amazon Web Services: Kinesis Firehose

What is Amazon Kinesis Firehose ?

Amazon Kinesis Firehose is the easiest way to load streaming data into AWS. With this new service you will be able to can capture and automatically load streaming data into Amazon S3 and Amazon Redshift. Enabling near real-time analytics with existing business intelligence tools and dashboards you’re already using today.

It is a fully managed service that automatically scales to match the throughput of your data and requires no ongoing administration. It can also batch, compress, and encrypt the data before loading it, minimizing the amount of storage used at the destination and increasing security. You can easily create a Firehose delivery stream from the AWS Management Console, configure it with a few clicks, and start sending data to the stream from hundreds of thousands of data sources to be loaded continuously to AWS – all in just a few minutes.

With Amazon Kinesis Firehose, you only pay for the amount of data you transmit through the service. There is no minimum fee or setup cost.

Amazon Kinesis Firehose

What offers Amazon Kinesis Firehose

Load New Data in Near Real-time

You can specify a batch size or batch interval to control how quickly data is uploaded to destinations. For example, you can set the batch interval to 60 seconds if you want to receive new data within 60 seconds of sending it to your delivery stream. Additionally, you can specify if data should be compressed. The service supports common compression algorithms including GZip, LZO, and Snappy. Batching and compressing data before uploading enables you to control how quickly you receive new data at the destinations.


Support for Multiple Data Destinations

Amazon Kinesis Firehose currently supports Amazon S3 and Amazon Redshift as destinations. You can specify the destination Amazon S3 bucket and optionally the Amazon Redshift table into which data should be loaded. You can have the same delivery stream load data into both Amazon S3 and Amazon Redshift simultaneously.

Easy Launch and Configuration

You can launch Amazon Kinesis Firehose and create a delivery stream to load data into Amazon S3, Amazon Redshift, or both with just a few clicks in the AWS Management Console. You can send data to the stream by calling the Firehose API, or running the Linux agent we provide on the data source. Amazon Kinesis Firehose then continuously loads the data into Amazon S3 and Amazon Redshift.

Elastic Scaling to Handle Varying Data Throughput 

Once launched, your delivery streams automatically scale up and down to handle gigabytes per second or more of input data rate, and maintain data latency at levels you specify for the stream. No intervention or maintenance is needed.

Pay-as-you-go pricing 

With Amazon Kinesis Firehose, you pay only for the volume of data you transmit through the service. There are no minimum fees or upfront commitments. You don’t need staff to operate, scale, and maintain infrastructure or custom applications to capture and load streaming data.

Read more …