Apache Kafka is an open source distributed streaming platform developed by LinkedIn and managed by the Apache software foundation.
Kafka is aimed to provide a high-throughput, low-latency, scalable, unified platform for handling real-time data streams.
Apache Kafka is based on the commit log principle i.e messages in Kafka goes for permanent record. Kafka allows pub/sub mechanism to produce and consume messages.
Here are tutorials to set up the Kafka cluster in the production environment.
1: What is Kafka
2: Setting Up Zookeeper Cluster for Kafka in AWS EC2
3: Setting up Multi-Broker Kafka in AWS EC2
4: Setting up Authentication in Multi-broker Kafka cluster in AWS EC2
5: Setting up Kafka management for Kafka cluster
6: Capacity Estimation for Kafka Cluster in production
7: Performance testing Kafka cluster
Let us know if there is any feedback/suggestion or bug in the setup tutorial.