Overview
This skill-centric two-day course comprises of 50% hands-on lab exercises and 50% lectures, and leads participants through a series of progressively advanced topics relating to working with Apache Kafka. Apache Kafka offers many significant benefits like high-scalability, fault tolerance, execution speed and fluid integration, making it an integral part of many enterprise data architectures. The course covers how to build effective streaming solutions using Kafka.
What You'll Learn
- Overview of streaming technologies
- Kafka concepts and architecture
- Programming using Kafka API
- Kafka streams
- Monitoring Kafka
- Tuning/troubleshooting Kafka
Curriculum
- Fast data
- Streaming architecture
- Lambda architecture
- Message queues
- Streaming processors
- Architecture
- Comparing Kafka with other queue systems (JMS/MQ)
- Kafka concepts: Message topics, partitions, brokers, producers, commit logs
- Kafka and Zookeeper
- Producing messages
- Consuming messages (Consumers and consumer groups)
- Message retention
- Scaling Kafka
- Lab: Getting Kafka up and running; Using Kafka utilities
- Configuring paramters
- Producer API (Sending messages to Kafka)
- Consumer API (Consuming messages from Kafka)
- Commits, Offsets, Seeking
- Schema with Avro
- Lab: Writing Kafka clients in Java; Benchmarking producer APIs
- Streams overview and architecture
- Streams use cases and comparison with other platforms
- Learning Kafka streaming concepts (KStream, KTable, KStore)
- KStreaming operations (transformations, filters, joins, aggregations)
- Lab: Kafka streaming labs
- Hardware/software requirements
- Deploying Kafka
- Configuration of brokers/topics/partitions/producers/consumers
- Security – How secure is the Kafka cluster
- Secure client communications (SASL, Kerberos)
- Monitoring tools
- Capacity planning – estimating usage and demand
- Troubleshooting – failure scenarios and recovery
- Monitoring Kafka
- Instrumenting with Metrics library
- Lab: Monitor Kafka cluster
- Instrument Kafka applications and monitor their performance
Build end-to-end application simulating web traffic and send metrics to Grafana
Who should attend
The course is highly recommended for –
- Software engineers
- Big data developers
- Big data platform engineers
- Spark developers
- Hadoop developers
- Scala developers
Prerequisites
Participants need to be experienced developers familiar with Java. They need to have experience working with databases. Participants must be able to navigate the Linux command line and have basic knowledge of Linux editors such as VI/nano for editing code.
