{"title":"Chapter 8: Backpressure","description":"","content":"# Chapter 8: Backpressure\n\n## Introduction\n\nIn previous chapters, we discussed how message queues buffer messages between producers and consumers. But what happens when producers consistently create messages faster than consumers can process them? The queue grows without bound, consuming memory and disk until the system crashes.\n\nThis is the problem backpressure solves. Backpressure is a mechanism that signals to producers to slow down when consumers cannot keep up. It is a feedback loop that prevents unbounded growth and keeps systems stable under load.\n\nThe term comes from fluid dynamics. When water cannot flow through a pipe fast enough, pressure builds up behind the obstruction. In software systems, backpressure serves a similar function: it pushes back against upstream producers when downstream consumers are overwhelmed.\n\nIn this chapter, you will learn:\n\n- What backpressure is and why it matters\n- Different backpressure strategies\n- How messaging systems implement backpressure\n- When and how to apply backpressure\n- Trade-offs between different approaches\n\nBy the end of this chapter, you will understand how to build systems that remain stable under varying loads by appropriately managing the flow of data between components.\n\n---\n\n## The Problem: Unbounded Queues\n\nWithout backpressure, a queue between a fast producer and a slow consumer grows indefinitely:\n\n\n```mermaid\nflowchart LR\n P[\"Producer
1000 msg/sec\"]:::primary --> Q[(Queue
Growing...)]:::red --> C[\"Consumer
500 msg/sec\"]:::orange\n\n Note1[\"Net growth: 500 msg/sec
Queue grows without limit\"]:::yellow\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef yellow fill:#ffd43b,stroke:#000,color:#000\n```\n\n\n### What Happens Next\n\n\n```mermaid\nflowchart TD\n Start[\"Queue growing\"]:::red --> Mem[\"Memory exhausted\"]:::red\n Mem --> Disk[\"Spill to disk\"]:::orange\n Disk --> DiskFull[\"Disk fills up\"]:::red\n DiskFull --> Crash[\"System crash
or
Message loss\"]:::red\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\nUnbounded queues eventually lead to:\n\n- Out of memory errors\n- Disk space exhaustion\n- Increasing latency (messages wait longer in queue)\n- System instability\n- Data loss when limits are hit\n\n### The Numbers\n\n\n```shell\nProducer: 1000 messages/sec\nConsumer: 800 messages/sec\nDifference: 200 messages/sec accumulating\n\nAfter 1 minute: 12,000 messages backlog\nAfter 1 hour: 720,000 messages backlog\nAfter 1 day: 17.3 million messages backlog\n\nIf each message is 1KB:\nAfter 1 day: 17.3 GB of backlog\n```\n\n\nEven small rate differences compound quickly.\n\n---\n\n## What is Backpressure?\n\nBackpressure is a feedback mechanism that slows down producers when consumers cannot keep up. Instead of letting the queue grow unbounded, the system signals producers to reduce their rate.\n\n\n```mermaid\nflowchart LR\n P[\"Producer\"]:::primary -->|\"Messages\"| Q[(Queue)]:::purple -->|\"Messages\"| C[\"Consumer\"]:::teal\n Q -->|\"Slow down!\"| P\n\n Note1[\"Feedback loop prevents
unbounded growth\"]:::yellow\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef yellow fill:#ffd43b,stroke:#000,color:#000\n```\n\n\n### The Analogy\n\nThink of a highway with an on-ramp:\n\n\n```mermaid\nflowchart LR\n subgraph NoBackpressure[\"Without Backpressure\"]\n Cars1[\"Cars enter freely\"]:::red --> Highway1[\"Highway\"]:::red --> Jam1[\"Traffic jam\"]:::red\n end\n\n subgraph WithBackpressure[\"With Backpressure (Ramp Metering)\"]\n Cars2[\"Metered entry\"]:::green --> Highway2[\"Highway\"]:::green --> Flow2[\"Traffic flows\"]:::green\n end\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nRamp meters control how fast cars enter the highway based on current traffic. Backpressure does the same for messages.\n\n---\n\n## Backpressure Strategies\n\nThere are several ways to implement backpressure, each with different trade-offs.\n\n### 1. Blocking (Synchronous Backpressure)\n\nThe producer blocks when the queue is full. It cannot send more until space is available.\n\n\n```mermaid\nsequenceDiagram\n participant P as Producer\n participant Q as Queue (Full)\n participant C as Consumer\n\n P->>Q: Send message\n Note over Q: Queue full!\n Note over P: Producer blocks...\n\n C->>Q: Pull message\n Note over Q: Space available\n Q-->>P: Unblock\n Note over P: Can send now\n P->>Q: Send message\n```\n\n\n**Characteristics:**\n\n\n| Aspect | Blocking |\n|--------|----------|\n| Producer impact | Stops completely when queue full |\n| Latency | Variable, depends on consumer |\n| Message loss | None |\n| Complexity | Low |\n| Use case | Internal services, same process |\n\n\n### 2. Dropping (Load Shedding)\n\nWhen the queue is full, new messages are rejected or dropped.\n\n\n```mermaid\nflowchart TB\n P[\"Producer\"]:::primary --> Check{\"Queue
full?\"}:::orange\n Check -->|\"No\"| Q[(Queue)]:::purple\n Check -->|\"Yes\"| Drop[\"Drop message\"]:::red\n\n Q --> C[\"Consumer\"]:::teal\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\n**Variations:**\n\n- **Drop newest:** Reject new messages when full\n- **Drop oldest:** Remove oldest messages to make room\n- **Drop random:** Randomly select messages to drop\n\n**Characteristics:**\n\n\n| Aspect | Dropping |\n|--------|----------|\n| Producer impact | Immediate rejection/error |\n| Latency | Predictable (bounded queue) |\n| Message loss | Yes, intentional |\n| Complexity | Low |\n| Use case | Metrics, real-time data, when freshness matters |\n\n\n### 3. Buffering with Limits\n\nAccept messages into a bounded buffer. When limits are reached, apply another strategy (block, drop, or signal).\n\n\n```mermaid\nflowchart LR\n P[\"Producer\"]:::primary --> Buffer[\"Bounded Buffer
Max: 10,000\"]:::purple --> C[\"Consumer\"]:::teal\n\n Buffer -->|\"80% full\"| Signal[\"Signal producer
to slow down\"]:::yellow\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef yellow fill:#ffd43b,stroke:#000,color:#000\n```\n\n\n**Characteristics:**\n\n\n| Aspect | Bounded Buffer |\n|--------|----------------|\n| Producer impact | Depends on what happens at limit |\n| Latency | Bounded by buffer size |\n| Message loss | Only if combined with dropping |\n| Complexity | Medium |\n| Use case | Most messaging systems |\n\n\n### 4. Rate Limiting\n\nControl the rate at which producers can send, independent of queue state.\n\n\n```mermaid\nflowchart LR\n P[\"Producer\"]:::primary --> RL[\"Rate Limiter
100 msg/sec\"]:::orange --> Q[(Queue)]:::purple --> C[\"Consumer\"]:::teal\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\n**Characteristics:**\n\n\n| Aspect | Rate Limiting |\n|--------|---------------|\n| Producer impact | Fixed rate cap |\n| Latency | Predictable |\n| Message loss | Only if producer generates above limit |\n| Complexity | Low |\n| Use case | API gateways, known capacity limits |\n\n\n### 5. Dynamic Rate Adjustment\n\nAdjust producer rate based on real-time signals from the queue or consumer.\n\n\n```mermaid\nflowchart TB\n P[\"Producer
(adjustable rate)\"]:::primary --> Q[(Queue)]:::purple --> C[\"Consumer\"]:::teal\n\n Q -->|\"Queue depth signal\"| Controller[\"Rate Controller\"]:::orange\n Controller -->|\"New rate: 800/sec\"| P\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n```\n\n\n**Characteristics:**\n\n\n| Aspect | Dynamic Adjustment |\n|--------|-------------------|\n| Producer impact | Varies based on system state |\n| Latency | Adapts to conditions |\n| Message loss | Minimal with good tuning |\n| Complexity | High |\n| Use case | Complex streaming systems |\n\n\n---\n\n## Comparison of Strategies\n\n\n| Strategy | Message Loss | Latency | Complexity | Best For |\n|----------|--------------|---------|------------|----------|\n| **Blocking** | None | High (variable) | Low | Same-process queues |\n| **Drop newest** | Yes | Low | Low | Real-time metrics |\n| **Drop oldest** | Yes | Low | Low | Latest-value scenarios |\n| **Bounded buffer** | Depends | Bounded | Medium | General messaging |\n| **Rate limiting** | Possible | Predictable | Low | Known capacity |\n| **Dynamic adjustment** | Minimal | Adaptive | High | Streaming platforms |\n\n\n---\n\n## Backpressure in Messaging Systems\n\n### Apache Kafka\n\nKafka implements backpressure through consumer group lag and producer configuration:\n\n\n```mermaid\nflowchart TB\n P[\"Producer\"]:::primary --> Topic[(Topic)]:::purple\n Topic --> P0[Partition 0]:::orange\n Topic --> P1[Partition 1]:::orange\n\n P0 --> CG[\"Consumer Group\"]:::teal\n P1 --> CG\n\n subgraph Signals[\"Backpressure Signals\"]\n Lag[\"Consumer lag monitoring\"]:::yellow\n Config[\"Producer buffer.memory limit\"]:::yellow\n Block[\"max.block.ms for blocking\"]:::yellow\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef yellow fill:#ffd43b,stroke:#000,color:#000\n```\n\n\n**Key configurations:**\n\n\n| Setting | Purpose |\n|---------|---------|\n| `buffer.memory` | Producer blocks when buffer is full |\n| `max.block.ms` | How long to block before throwing exception |\n| Consumer lag | Monitor to detect slow consumers |\n\n\n### RabbitMQ\n\nRabbitMQ uses credit-based flow control:\n\n\n```mermaid\nflowchart LR\n P[\"Producer\"]:::primary -->|\"Send (uses credit)\"| Q[(Queue)]:::purple --> C[\"Consumer\"]:::teal\n Q -->|\"Replenish credit\"| P\n\n Note1[\"Producer can only send
if it has credit\"]:::yellow\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef yellow fill:#ffd43b,stroke:#000,color:#000\n```\n\n\n**Flow control triggers:**\n\n- Memory high watermark (default 40% of RAM)\n- Disk space low\n- Connection overloaded\n\nWhen triggered, RabbitMQ blocks producers from publishing.\n\n### Amazon SQS\n\nSQS uses message limits and visibility timeout for implicit backpressure:\n\n\n```mermaid\nflowchart TB\n P[\"Producer\"]:::primary --> SQS[(SQS Queue)]:::purple\n\n SQS --> Check{\"In-flight
messages
> 120,000?\"}:::orange\n Check -->|\"Yes\"| Block[\"OverLimit error
Throttled\"]:::red\n Check -->|\"No\"| C[\"Consumer\"]:::teal\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\n**SQS limits:**\n\n- Max 120,000 in-flight messages (standard queue)\n- Max 20,000 in-flight messages (FIFO queue)\n- Messages exceeding limits return `OverLimit` error\n\n---\n\n## Implementing Backpressure\n\n### Pattern 1: Bounded Queue with Blocking\n\n\n```shell\nqueue_capacity = 10000\n\nsend(message):\n while queue.size >= queue_capacity:\n wait(100ms) // Block until space available\n queue.add(message)\n```\n\n\nSimple but can cause producer thread starvation.\n\n### Pattern 2: Bounded Queue with Error Response\n\n\n```shell\nqueue_capacity = 10000\n\nsend(message):\n if queue.size >= queue_capacity:\n return Error(\"Queue full, try again later\")\n queue.add(message)\n return Success\n```\n\n\nProducer must handle the error (retry with backoff).\n\n### Pattern 3: High/Low Watermarks\n\n\n```mermaid\nflowchart TB\n subgraph Queue[\"Queue State\"]\n High[\"High watermark: 8000\"]:::red\n Level[\"Current: ???\"]:::purple\n Low[\"Low watermark: 2000\"]:::green\n end\n\n Level -->|\"> 8000\"| Pause[\"Signal: PAUSE\"]:::red\n Level -->|\"< 2000\"| Resume[\"Signal: RESUME\"]:::green\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\n\n```shell\nsend(message):\n queue.add(message)\n\n if queue.size > HIGH_WATERMARK:\n signal_producers(PAUSE)\n\nconsume():\n message = queue.take()\n process(message)\n\n if queue.size < LOW_WATERMARK:\n signal_producers(RESUME)\n```\n\n\nThis provides hysteresis, avoiding rapid pause/resume oscillation.\n\n### Pattern 4: Credit-Based Flow Control\n\n\n```mermaid\nsequenceDiagram\n participant P as Producer\n participant C as Consumer\n\n Note over C: Grant 100 credits\n C->>P: Credits: 100\n\n P->>C: Message 1 (credit: 99)\n P->>C: Message 2 (credit: 98)\n P->>C: Message 3 (credit: 97)\n\n Note over C: Process messages\n Note over C: Grant more credits\n C->>P: Credits: +50 (total: 147)\n\n Note over P: Can send 147 more messages\n```\n\n\nProducer can only send if it has credits. Consumer controls the flow by granting credits.\n\n---\n\n## When to Apply Backpressure\n\n### Apply Backpressure When:\n\n1. **Producers consistently outpace consumers**\n\n- Sustained rate mismatch\n- Growing queue depth over time\n\n1. **Memory/disk resources are constrained**\n\n- Cannot afford unbounded growth\n- Need predictable resource usage\n\n1. **Latency bounds matter**\n\n- Messages sitting in queue too long become stale\n- SLA requirements on end-to-end latency\n\n1. **System stability is critical**\n\n- Cannot afford crashes from resource exhaustion\n- Need graceful degradation under load\n\n### Do Not Apply Backpressure When:\n\n1. **Temporary bursts are expected**\n\n- If consumer will catch up during quiet periods\n- Use queue as buffer for smoothing\n\n1. **Dropping messages is acceptable**\n\n- Metrics, telemetry where latest value matters\n- Use drop-oldest strategy instead\n\n1. **Producers cannot slow down**\n\n- External event sources (IoT sensors)\n- Use buffering with overflow handling\n\n---\n\n## Monitoring for Backpressure\n\n### Key Metrics\n\n\n| Metric | What It Tells You |\n|--------|-------------------|\n| **Queue depth** | Current backlog size |\n| **Producer rate** | How fast messages arrive |\n| **Consumer rate** | How fast messages are processed |\n| **Queue growth rate** | producer rate - consumer rate |\n| **Message age** | How long messages wait in queue |\n| **Backpressure events** | How often producers are slowed |\n\n\n### Dashboard Example\n\n\n```mermaid\nflowchart TB\n subgraph Metrics[\"Backpressure Dashboard\"]\n M1[\"Queue Depth
██████░░░░ 6,000/10,000\"]:::orange\n M2[\"Producer Rate
1,200 msg/sec\"]:::primary\n M3[\"Consumer Rate
1,000 msg/sec\"]:::teal\n M4[\"Net Growth
+200 msg/sec ⚠️\"]:::yellow\n M5[\"Oldest Message
45 seconds\"]:::orange\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef yellow fill:#ffd43b,stroke:#000,color:#000\n```\n\n\n### Alert Thresholds\n\n\n```shell\nWarning:\n - Queue depth > 50% capacity\n - Message age > 30 seconds\n - Backpressure events > 10/minute\n\nCritical:\n - Queue depth > 80% capacity\n - Message age > 2 minutes\n - Sustained backpressure > 5 minutes\n```\n\n\n---\n\n## Interview Questions\n\n**Q: What is backpressure and why is it important?**\n\nA: Backpressure is a feedback mechanism that slows down producers when consumers cannot keep up with the message rate. Without backpressure, when producers are faster than consumers, queues grow without bound until memory or disk is exhausted, leading to system crashes. Backpressure prevents this by signaling producers to reduce their rate, maintaining system stability. It is essential for building reliable systems that handle varying loads gracefully.\n\n**Q: What are the main strategies for implementing backpressure?**\n\nA: The main strategies are blocking, where producers stop and wait when the queue is full; dropping, where messages are rejected or removed when capacity is exceeded; bounded buffers with limits that trigger other actions when reached; rate limiting, which caps producer rate regardless of queue state; and dynamic rate adjustment, which adapts producer rate based on real-time signals. Each has trade-offs. Blocking guarantees no message loss but can stall producers. Dropping maintains latency but loses messages. The choice depends on whether message loss is acceptable and what latency bounds are required.\n\n**Q: How does Kafka handle backpressure?**\n\nA: Kafka handles backpressure through several mechanisms. On the producer side, the `buffer.memory` setting limits how much unsent data the producer buffers. If the buffer fills up, the producer blocks for up to `max.block.ms` before throwing an exception. On the consumer side, consumer lag, which is the difference between the latest offset and the consumer's position, serves as a signal that consumers are falling behind. Operations teams monitor consumer lag to detect slow consumers. Kafka does not automatically slow producers based on consumer lag, but this can be implemented at the application level.\n\n**Q: When would you choose to drop messages instead of applying backpressure?**\n\nA: I would choose to drop messages when the data is time-sensitive and old data is less valuable than current data, like real-time metrics or stock prices. Also when the system cannot control producer rate, such as external IoT sensors that send data regardless of consumer state. When latency is more important than completeness, because keeping queues bounded ensures predictable latency. When the producer has no way to slow down meaningfully. In these cases, I would typically drop the oldest messages to keep the most recent data, or drop random messages to maintain statistical validity for aggregation use cases.\n\n---\n\n## Summary\n\nBackpressure is a critical mechanism for maintaining system stability when producers outpace consumers:\n\n**The problem:**\n\n- Without backpressure, queues grow unbounded\n- Memory and disk exhaustion cause crashes\n- Even small rate mismatches compound over time\n\n**Strategies:**\n\n- **Blocking:** Producer waits for space (no message loss)\n- **Dropping:** Messages rejected when full (latency bounded)\n- **Bounded buffers:** Fixed limits with configurable overflow behavior\n- **Rate limiting:** Fixed producer caps\n- **Dynamic adjustment:** Adapt based on real-time feedback\n\n**Implementation patterns:**\n\n- High/low watermarks prevent oscillation\n- Credit-based flow control for fine-grained management\n- Monitor queue depth, rates, and message age\n\n**Key insight:** Backpressure is about system stability and predictability. Choose your strategy based on whether message loss is acceptable and what latency guarantees you need. Most production systems use bounded buffers with monitoring and alerting.\n\nIn the final chapter of this section, we will explore a different approach to asynchronous communication: Change Data Capture (CDC). Instead of applications explicitly publishing events, CDC captures changes from database logs, enabling reliable event streaming without modifying application code.\n\n---\n\n## References\n\n- [Reactive Streams Specification](undefined) - Backpressure in reactive systems\n- [Designing Data-Intensive Applications](undefined) by Martin Kleppmann - Flow control in distributed systems\n- [Kafka Documentation: Producer Configs](undefined) - Kafka backpressure settings\n- [RabbitMQ Flow Control](undefined) - RabbitMQ credit-based flow control\n- [Building Reactive Microservices](undefined) - Reactive patterns and backpressure\n- [Little's Law](undefined) - Understanding queue dynamics","pageType":"system-design"}

Chapter 8: Backpressure

Ashish Pratap Singh

Get Premium