Apache Kafka in Action

Apache Kafka, start to finish. Apache Kafka in Action: From basics to production guides you through the concepts and skills you‘ll need to deploy and administer Kafka for data pipelines, event-driven applications, and other systems that process data streams from multiple sources. Authors Anatoly Zelenin and Alexander Kropp have spent years using Kafka in real-world production environments. In this guide, they reveal their hard-won expert insights to help you avoid common Kafka pitfalls and challenges. Inside Apache Kafka in Action you‘ll discover: Apache Kafka from the ground up Achieving reliability and performance Troubleshooting Kafka systems Operations, governance, and monitoring Kafka use cases, patterns, and anti-patterns Clear, concise, and practical, Apache Kafka in Action is written for IT operators, software engineers, and IT architects working with Kafka every day. Chapter by chapter, it guides you through the skills you need to deliver and maintain reliable and fault-tolerant data-driven applications. Foreword by Adam Bellemare. About the technology Apache Kafka is the gold standard streaming data platform for real-time analytics, event sourcing, and stream processing. Acting as a central hub for distributed data, it enables seamless flow between producers and consumers via a publish-subscribe model. Kafka easily handles millions of events per second, and its rock-solid ensures high fault tolerance and smooth scalability. About the book Apache Kafka in Action is a practical guide for IT professionals who are integrating Kafka into data-intensive applications and infrastructures. The book covers everything from Kafka fundamentals to advanced operations, with interesting visuals and real-world examples. Readers will learn to set up Kafka clusters, produce and consume messages, handle real-time streaming, and integrate Kafka into enterprise systems. This easy-to-follow book emphasizes building reliable Kafka applications and taking advantage of its distributed architecture for scalability and resilience. What‘s inside Master Kafka‘s distributed streaming capabilities Implement real-time data solutions Integrate Kafka into enterprise environments Build and manage Kafka applications Achieve fault tolerance and scalability About the reader For IT operators, software architects and developers. No experience with Kafka required. About the author Anatoly Zelenin is a Kafka expert known for workshops across Europe, especially in banking and manufacturing. Alexander Kropp specializes in Kafka and Kubernetes, contributing to cloud platform and monitoring. Table of Contents Part 1 1 Introduction to Apache Kafka 2 First steps with Kafka Part 2 3 Exploring Kafka topics and messages 4 Kafka as a distributed log 5 Reliability 6 Performance Part 3 7 Cluster management 8 Producing and persisting messages 9 Consuming messages 10 Cleaning up messages Part 4 11 Integrating external systems with Kafka Connect 12 Stream processing 13 Governance 14 Kafka reference architecture 15 Kafka monitoring and alerting16 Disaster management 17 Comparison with other technologies 18 Kafka‘s role in modern enterprise architectures Appendixes A Setting up a Kafka test environment B Monitoring setup