Course Overview

Apache Kafka provides a unified, high-throughput, low-latency platform to handle real-time data feeds. This course will show you how to use Kafka efficiently, and contains practical solutions to the common problems that developers and administrators usually face while working with it. 

This course will teach you how to:

  • Install and configure Apache Kafka 1.0 to get optimal performance
  • Create and configure Kafka Producers and Consumers
  • Operate your Kafka clusters efficiently by implementing the mirroring technique
  • Work with the new Confluent platform and Kafka streams, and achieve high availability with Kafka
  • Monitor Kafka using tools such as Graphite and Ganglia
  • Integrate Kafka with third-party tools such as Elasticsearch, Logstash, Apache Hadoop, Apache Spark, and more

Pre-requisites

Required:  Basic computer skills and Internet Access.  An understanding of Hadoop and MapReduce is required.

Recommended:  Basic Analytic or Programming, or Operating System Skills

Number of Days and Format

2 Training Days

Instructor-Led In Person / Remote / Training on Demand 

Course curriculum

  • 1
    Configuring Kafka
    • Installing Kafka
    • Lab 0 - Install on Ubuntu 16.04
    • LAB01. Installing Kafka
    • Running Kafka
    • LAB02. Running Kafka
    • Configuring Kafka brokers
    • LAB03. Configuring Kafka brokers
    • Configuring Kafka topics
    • LAB04. Configuring Kafka topics
    • Creating a message console producer
    • LAB05. Creating a message console producer
    • Creating a message console consumer
    • LAB06. Creating a message console consumer
    • Configuring the broker settings
    • LAB07. Configuring the broker settings
    • Configuring threads and performance
    • LAB08. Configuring threads and performance
    • Configuring the log settings
    • LAB09. Configuring the log settings
    • Configuring the replica settings
    • LAB10. Configuring the replica settings
    • Configuring the ZooKeeper settings
    • LAB11. Configuring the ZooKeeper settings
    • Configuring other miscellaneous parameters
    • Configuring other miscellaneous parameters
    • LAB12. Configuring other miscellaneous parameters
  • 2
    Kafka Clusters
    • Configuring a single-node single-broker cluster – SNSB
    • LAB01. Configuring a single-node single-broker cluster – SNSB
    • SNSB – creating a topic, producer, and consumer
    • LAB02. SNSB – creating a topic, producer, and consumer
    • Configuring a single-node multiple-broker cluster – SNMB
    • LAB03. Configuring a single-node multiple-broker cluster – SNMB
    • SNMB – creating a topic, producer, and consumer
    • LAB04. SNMB – creating a topic, producer, and consumer
    • Configuring a multiple-node multiple-broker cluster – MNMB
    • LAB05. Configuring a multiple-node multiple-broker cluster – MNMB
  • 3
    Message Validation
    • Modeling the events
    • LAB01. Modeling the events
    • Setting up the project
    • LAB02. Setting up the project
    • Reading from Kafka
    • LAB03. Reading from Kafka
    • Writing to Kafka
    • LAB04. Writing to Kafka
    • Running ProcessingApp
    • LAB05. Running ProcessingApp
    • Coding the validator
    • LAB06. Coding the validator
    • Running the validator
    • LAB07. Running the validator
  • 4
    Message Enrichment
    • Geolocation extractor
    • LAB01. Geolocation extractor
    • Geolocation enricher
    • LAB02. Geolocation enricher
    • Currency price extractor
    • LAB03. Currency price extractor
    • Currency price enricher
    • LAB04. Currency price enricher
    • Running the currency price enricher
    • LAB05. Running the currency price enricher
    • Modeling the events
    • LAB06. Modeling the events
    • Setting up the project
    • LAB07. Setting up the project
    • Open weather extractor
    • LAB08. Open weather extractor
    • Location temperature enricher
    • LAB09. Location temperature enricher
    • Running the location temperature enricher
    • LAB10. Running the location temperature enricher
  • 5
    The Confluent Platform
    • Installing the Confluent Platform
    • LAB01. Installing the Confluent Platform
    • Using Kafka operations
    • Using Kafka operations
    • LAB02. Using Kafka operations
    • Monitoring with the Confluent Control Center
    • LAB03. Monitoring with the Confluent Control Center
    • Using the Schema Registry
    • LAB04. Using the Schema Registry
    • Using the Kafka REST Proxy
    • LAB05. Using the Kafka REST Proxy
    • Using Kafka Connect
    • LAB06. Using Kafka Connect
  • 6
    Kafka Streams
    • Setting up the project
    • LAB01. Setting up the project
    • Running the streaming application
    • LAB02. Running the streaming application
  • 7
    Managing Kafka
    • Managing consumer groups
    • Dumping log segments
    • Importing ZooKeeper offsets
    • Using the GetOffsetShell
    • Using the JMX tool
    • Using the MirrorMaker tool
    • Replaying log producer
    • Using state change log merger
  • 8
    Operating Kafka
    • Adding or removing topics
    • Modifying message topics
    • Implementing a graceful shutdown
    • Balancing leadership
    • Expanding clusters
    • Increasing the replication factor
    • Decommissioning brokers
    • Checking the consumer position
  • 9
    Monitoring and Security
    • Monitoring server statistics
    • Monitoring producer statistics
    • Monitoring consumer statistics
    • Connecting with the help of Graphite
    • Monitoring with the help of Ganglia
    • Implementing authentication using SSL
    • Implementing authentication using SASL/Kerberos
  • 10
    Third-Party Tool Integration
    • Moving data between Kafka nodes with Flume
    • Writing to an HDFS cluster with Gobblin
    • Moving data from Kafka to Elastic with Logstash
    • Connecting Spark streams and Kafka
    • Ingesting data from Kafka to Storm
    • Pushing data from Kafka to Elastic
    • Inserting data from Kafka to SolrCloud
    • Building a Kafka producer with Akka
    • Building a Kafka consumer with Akka
    • Storing data in Cassandra
    • Running Kafka on Mesos
    • Reading Kafka with Apache Beam
    • Writing to Kafka from Apache Beam
  • 11
    Resources
    • Downloads

Instructor(s)