MSK (Managed Streaming for Apache Kafka)
💡 Definition
Amazon MSK is a fully managed service that makes it easy for you to build and run applications that use Apache Kafka to process streaming data.
🔑 Key Concepts
- Apache Kafka: A popular open-source distributed event streaming platform.
- Managed Service: AWS handles the provisioning, configuration, and maintenance of Zookeeper and Kafka clusters.
- Migration: Ideal for customers already using Kafka on-premises who want to migrate to AWS without code changes.
⚙️ How it Works
It works just like a self-managed Kafka cluster but without the operational overhead. You produce data to MSK topics and consume data using Kafka consumers (EC2, Lambda, containers).
🎯 Use Cases
- Lift and Shift: Migrating existing Kafka workloads to the cloud.
- Streaming Data Pipelines: Building pipes to move data between systems.
💰 Pricing Model
- Broker Hours: Charged per broker instance hour.
- Storage: Charged per GB-month of storage.
📝 Exam Tips (CLF-C02)
- Keyword: "Apache Kafka".
- Managed service for Kafka.
- Alternative to Kinesis (Kinesis is AWS proprietary; MSK is open-source Kafka).