Chapter 8: Streaming Architectures

In this section

• 8.1 Introduction to Streaming
  • Understanding Streaming in Computing: Real-Time Data Processing
    Explore the concept of streaming in computing, its components, data flow, and real-world applications. Learn how streaming differs from batch processing and its evolution over time.
  • Benefits of Streaming Architectures: Real-Time Data Processing, Low Latency, and Scalability
    Explore the benefits of streaming architectures, including real-time data processing, low latency, scalability, and more. Learn how these systems enhance user experiences and integrate seamlessly with other technologies.
  • Key Concepts and Terminology in Streaming Architectures
    Explore the essential concepts and terminology in streaming architectures, including data streams, stream processing, windowing, and more.
• 8.2 Implementing Streaming Systems
  • Apache Kafka Streams: Building Real-Time Stream Processing Applications
    Explore Apache Kafka Streams, a powerful client library for building real-time, scalable, and fault-tolerant stream processing applications within the Kafka ecosystem. Learn about setting up Kafka Streams, defining stream processing topologies, and implementing stateful operations with practical examples.
  • Apache Flink: High-Throughput Stream Processing for Event-Driven Architectures
    Explore Apache Flink, an open-source stream processing framework for high-throughput, low-latency data processing, with support for event time and stateful computations. Learn about its setup, programming model, and robust features for building scalable event-driven systems.
  • Comparing Streaming Frameworks: Apache Kafka Streams vs. Apache Flink and More
    Explore the key criteria for comparing streaming frameworks, including Apache Kafka Streams, Apache Flink, and others, focusing on performance, scalability, and ease of use.
• 8.3 Stream Processing Patterns
  • Stateful vs. Stateless Processing in Stream Architectures
    Explore the differences between stateful and stateless processing in stream architectures, including use cases, advantages, implementation considerations, and best practices.
  • Windowing and Aggregations in Stream Processing
    Explore the intricacies of windowing and aggregations in stream processing, including types of windows, implementation strategies, and practical examples using Apache Flink.
  • Event Time vs. Processing Time in Stream Processing
    Explore the critical differences between event time and processing time in stream processing, their advantages, trade-offs, and implementation in frameworks like Apache Flink and Kafka Streams.
• 8.4 Integrating Streaming with EDA
  • Real-Time Event Handling in Streaming Architectures
    Explore the integration of real-time event handling within streaming architectures, focusing on immediate processing, integration with event brokers, and optimizing for low latency.
  • Data Pipelines and ETL in Streaming Architectures
    Explore the integration of data pipelines and ETL processes within streaming architectures, focusing on real-time data transformation, enrichment, and optimization for event-driven systems.
  • Ensuring Consistency and Reliability in Streaming Event-Driven Architectures
    Explore strategies for ensuring consistency and reliability in streaming event-driven architectures, focusing on exactly-once semantics, idempotent processing, and state store consistency.