{"title":"Traffic Patterns","description":"","content":"On November 11, 2020, Alibaba processed $74 billion in Singles’ Day sales and hit a peak of 583,000 orders per second. The important detail is that the peak lasted only minutes. Provisioning for that peak all year would waste enormous capacity.\n\nTotal load is only half the story. The distribution over time matters just as much. Traffic is shaped by human behavior and events: daily rhythms, weekends, promotions, and sudden spikes when attention shifts.\n\nA system doing 10 million requests per day might need only ~100 RPS if traffic is steady, or ~1,000 RPS if most of it arrives in a short evening window. Same daily volume, very different capacity requirements.\n\nUnderstanding traffic patterns lets you plan for real peaks, tune auto-scaling, and choose architectures that match how your users actually arrive.\n\n---\n\n# Why Traffic Patterns Matter\n\nConsider two services, both handling 10 million requests per day:\n\n**Service A:** Traffic is perfectly uniform at 116 RPS constant throughout the day.\n\n**Service B:** Traffic peaks at 1,000 RPS for 2 hours each evening, drops to 20 RPS overnight.\n\n\n```mermaid\nflowchart LR\n subgraph ServiceB[\"Service B: Peak-Heavy Traffic\"]\n B1[12 AM
20 RPS]:::green\n B2[6 AM
50 RPS]:::green\n B3[12 PM
200 RPS]:::orange\n B4[6 PM
1000 RPS]:::red\n B5[11 PM
300 RPS]:::orange\n end\n\t\n subgraph ServiceA[\"Service A: Uniform Traffic\"]\n A1[12 AM
116 RPS]:::teal\n A2[6 AM
116 RPS]:::teal\n A3[12 PM
116 RPS]:::teal\n A4[6 PM
116 RPS]:::teal\n A5[11 PM
116 RPS]:::teal\n end\t\n\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\nSame daily volume, but vastly different infrastructure needs:\n\n\n| Aspect | Service A (Uniform) | Service B (Peak-Heavy) |\n|--------|---------------------|------------------------|\n| **Minimum capacity needed** | 150 RPS (with buffer) | 1,200 RPS (with buffer) |\n| **Average utilization** | ~77% | ~8% if static provisioning |\n| **Infrastructure cost** | Low (right-sized) | High unless auto-scaling |\n| **Scaling complexity** | Simple static provisioning | Requires auto-scaling or scheduled scaling |\n| **Risk of outage** | Low (predictable) | High if peak underestimated |\n\n\nService B costs 8x more than Service A if you provision statically for peak load. But with proper auto-scaling, Service B can approach Service A's cost efficiency by scaling down during off-peak hours. This is why understanding your traffic pattern is the first step in cost-effective capacity planning.\n\n### Pattern Impact on Architecture\n\nDifferent patterns favor different architectural approaches:\n\n\n```mermaid\nflowchart TD\n Pattern[Traffic Pattern]:::primary\n\n Pattern --> Steady[Steady/Uniform]:::teal\n Pattern --> Daily[Daily Peaks]:::orange\n Pattern --> Bursty[Bursty/Unpredictable]:::red\n Pattern --> Event[Event-Driven]:::purple\n Pattern --> Geo[Geographic Waves]:::green\n\n Steady --> StaticArch[Static provisioning
Simple architecture]:::teal\n Daily --> AutoArch[Auto-scaling
Pre-warming]:::orange\n Bursty --> OverArch[Over-provision
Aggressive auto-scale]:::red\n Event --> QueueArch[Queue-based
Graceful degradation]:::purple\n Geo --> CDNArch[Multi-region
CDN-heavy]:::green\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\n\n| Traffic Pattern | Architectural Implication | Example |\n|-----------------|--------------------------|---------|\n| **Steady** | Static provisioning, simple architecture | Internal APIs, batch processing |\n| **Daily peaks** | Auto-scaling, pre-warming before peaks | Consumer apps, social media |\n| **Bursty/unpredictable** | Over-provision or aggressive auto-scaling | News sites, viral content |\n| **Event-driven spikes** | Queue-based architecture, graceful degradation | E-commerce sales, product launches |\n| **Geographic waves** | Multi-region, CDN-heavy, follow-the-sun | Global SaaS, streaming services |\n\n\n---\n\n# Common Traffic Patterns\n\nMost applications exhibit a combination of patterns at different time scales. Understanding each pattern type helps you design appropriate capacity strategies.\n\n### Daily Patterns\n\nHuman behavior creates predictable daily cycles. People wake up, work, eat, relax, and sleep at roughly similar times within a timezone. This creates the most common and predictable pattern you will encounter.\n\n\n```mermaid\nflowchart TB\n subgraph Daily[\"Typical Consumer App Daily Pattern\"]\n D1[12-6 AM
Trough
0.3x]:::teal\n D2[6-9 AM
Rising
0.8x]:::green\n D3[9-12 PM
Plateau
1.2x]:::orange\n D4[12-2 PM
Lunch Dip
1.0x]:::green\n D5[2-6 PM
Steady
1.3x]:::orange\n D6[6-11 PM
Peak
2.5x]:::red\n D7[11 PM-12
Falling
0.8x]:::green\n end\n\n D1 --> D2 --> D3 --> D4 --> D5 --> D6 --> D7\n\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\nThe multipliers shown (0.3x, 2.5x) are relative to daily average. A consumer app with 100 RPS average might see 30 RPS at 4 AM and 250 RPS at 8 PM.\n\n**Daily pattern characteristics by application type:**\n\n\n| Application | Peak Time | Trough Time | Peak:Trough Ratio | Pattern Shape |\n|-------------|-----------|-------------|-------------------|---------------|\n| **Enterprise SaaS** | 10 AM - 4 PM local | Nights, weekends | 5:1 to 10:1 | Sharp business hours |\n| **Consumer social** | 7 PM - 11 PM local | 3 AM - 6 AM local | 3:1 to 5:1 | Evening mountain |\n| **Gaming** | 7 PM - 12 AM local | 6 AM - 2 PM local | 4:1 to 8:1 | Late evening peak |\n| **Streaming video** | 8 PM - 11 PM local | 4 AM - 8 AM local | 5:1 to 10:1 | Prime time spike |\n| **E-commerce** | 12 PM and 8 PM local | 3 AM - 6 AM local | 3:1 to 5:1 | Dual peaks |\n| **Financial trading** | Market hours | After hours | 50:1+ | Extreme concentration |\n\n\nThe peak:trough ratio directly impacts capacity decisions. A 50:1 ratio for financial trading means you either massively over-provision or implement aggressive scaling. A 3:1 ratio is much more manageable with standard auto-scaling.\n\n### Weekly Patterns\n\nDay-of-week variations layer on top of daily patterns. These are especially pronounced for applications with distinct business versus leisure use cases.\n\n\n```mermaid\nflowchart TB\n subgraph Week[\"B2B SaaS Weekly Pattern\"]\n Mon[Monday
120%]:::red\n Tue[Tuesday
110%]:::orange\n Wed[Wednesday
100%]:::green\n Thu[Thursday
105%]:::orange\n Fri[Friday
85%]:::green\n Sat[Saturday
20%]:::teal\n Sun[Sunday
15%]:::teal\n end\n\n Mon --> Tue --> Wed --> Thu --> Fri --> Sat --> Sun\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n```\n\n\n\n| Day Pattern | Description | Examples |\n|-------------|-------------|----------|\n| **Weekday dominant** | Monday-Friday traffic is 3-5x weekend | Enterprise apps, B2B SaaS, professional tools |\n| **Weekend dominant** | Saturday-Sunday traffic is 2-3x weekday | Gaming, entertainment, travel booking |\n| **Flat** | Little day-of-week variation | Infrastructure services, messaging, email |\n| **Monday spike** | Heavy Monday morning traffic | Enterprise tools (catching up), email |\n| **Friday dip** | Lower Friday afternoon traffic | Enterprise (early departures) |\n| **Sunday dip** | Lowest day of the week | Most business applications |\n\n\nUnderstanding weekly patterns affects:\n\n- **Maintenance windows:** Schedule during lowest traffic (Sunday night for B2B, Tuesday night for gaming)\n- **Deployment timing:** Avoid deploying before peak days\n- **Auto-scaling schedules:** Can pre-configure weekly patterns\n- **Cost optimization:** Scale down aggressively on low-traffic days\n\n### Seasonal Patterns\n\nLonger cycles driven by seasons, holidays, and business cycles create significant capacity variations throughout the year.\n\n\n```mermaid\nflowchart TB\n subgraph Year[\"E-commerce Annual Traffic Pattern\"]\n Q1[Jan-Mar
80-90%]:::teal\n Q2[Apr-Jun
85-95%]:::green\n Q3[Jul-Sep
90-100%]:::green\n Q4[Oct-Dec
150-250%]:::red\n end\n\n Q1 --> Q2 --> Q3 --> Q4\n\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n\n| Seasonal Cycle | Peak Period | Industries Affected | Planning Lead Time |\n|----------------|-------------|---------------------|-------------------|\n| **Holiday shopping** | Nov-Dec (Black Friday, Christmas) | E-commerce, retail, shipping | 3-6 months |\n| **Tax season** | Jan-April (US), varies by country | Financial software, accounting | 2-3 months |\n| **Back to school** | August-September | Education, retail, office supplies | 2-3 months |\n| **Summer lull** | June-August | B2B, enterprise (vacations) | Reduce capacity |\n| **Year-end surge** | December | B2B (budget spending), enterprise | 2-3 months |\n| **Weather-driven** | Varies by season | Travel, food delivery, outdoor retail | 1-2 months |\n\n\n#### **E-commerce detailed annual pattern**\n\n\n| Month | Traffic Index | Key Events | Planning Notes |\n|-------|---------------|------------|----------------|\n| January | 80 | Post-holiday lull | Good for maintenance |\n| February | 85 | Valentine's Day | Minor bump mid-month |\n| March-April | 90 | Spring shopping, Easter | Gradual increase |\n| May | 95 | Mother's Day | Weekend spike |\n| June-August | 85 | Summer lull | Vacation season |\n| September | 100 | Back to school | Return to baseline |\n| October | 110 | Halloween, early holiday | Building momentum |\n| November | 200-250 | Black Friday, Cyber Monday | Peak preparation critical |\n| December | 180 | Holiday shopping | Sustained high traffic |\n\n\nThe November spike of 200-250% is the critical planning point for e-commerce. If your baseline capacity handles September traffic, you need 2-2.5x that capacity for Black Friday. This requires months of preparation: load testing, infrastructure provisioning, cache warming, and team coordination.\n\n### Event-Driven Patterns\n\nExternal events cause traffic spikes that may or may not be predictable. These are often the most challenging patterns to handle because they can be extreme and sudden.\n\n\n```mermaid\nflowchart TD\n subgraph Events[\"Event-Driven Traffic Spikes\"]\n Planned[Planned Events
Predictable timing]:::green\n Unplanned[Unplanned Events
No warning]:::red\n end\n\n Planned --> Launch[Product Launch
5-50x, weeks notice]:::green\n Planned --> Marketing[Marketing Campaign
2-10x, days notice]:::green\n Planned --> Scheduled[Scheduled Event
Varies, known date]:::green\n\n Unplanned --> Viral[Viral Content
10-100x+, minutes]:::red\n Unplanned --> News[News Mention
5-20x, hours]:::red\n Unplanned --> Outage[Outage Recovery
2-5x thundering herd]:::red\n\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n\n| Event Type | Predictability | Lead Time | Traffic Multiple | Duration |\n|------------|---------------|-----------|------------------|----------|\n| **Planned marketing** | High | Days to weeks | 2-10x | Hours to days |\n| **Product launch** | High | Weeks to months | 5-50x | Hours to days |\n| **Scheduled events** | High | Known in advance | Varies by event | Event duration |\n| **TV/media mentions** | Medium | Sometimes hours | 5-20x | Minutes to hours |\n| **News/viral** | Low | Minutes to hours | 10-100x+ | Hours to days |\n| **Outage recovery** | Low | None | 2-5x (thundering herd) | Minutes |\n\n\n#### **Case study: Slack's Super Bowl ad (2021)**\n\nWhen Slack ran a Super Bowl commercial, they expected traffic to spike immediately after the ad aired. Their preparation included:\n\n1. **Pre-scaling:** Infrastructure scaled 2 hours before the game\n2. **Load testing:** Tested at 5x expected peak weeks before\n3. **Circuit breakers:** Implemented graceful degradation for non-critical features\n4. **War room:** Engineers on standby throughout the game\n5. **Runbooks:** Documented responses to various failure scenarios\n6. **Post-event:** Gradual scale-down over 24 hours\n\nThe result: they handled a 5x traffic spike without incidents. The key was treating the event like a planned deployment with full preparation and monitoring.\n\n### The Thundering Herd Problem\n\nA special pattern deserves attention: the thundering herd. This occurs when many clients simultaneously retry after an outage or when a cache expires, creating a sudden spike that can trigger cascading failures.\n\n\n```mermaid\nflowchart TD\n Outage[System Outage
or Cache Expiry]:::red --> Queue[Requests Queue
During Downtime]:::orange\n Queue --> Recovery[System Recovers]:::green\n Recovery --> Herd[Thundering Herd
All requests hit at once]:::red\n Herd --> Overload[System Overloads
→ Outage Again]:::red\n\n Overload -.-> Outage\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\n#### **Thundering herd scenarios**\n\n\n| Trigger | Mechanism | Prevention |\n|---------|-----------|------------|\n| **Service recovery** | Queued requests flood recovered service | Gradual connection restoration |\n| **Cache expiry** | Many keys expire simultaneously | Jittered TTLs, staggered expiry |\n| **Scheduled jobs** | Many jobs start at same time (top of hour) | Random offset, distributed scheduling |\n| **Client retries** | All clients retry at same interval | Exponential backoff with jitter |\n| **DNS TTL expiry** | All clients refresh DNS simultaneously | Staggered TTLs |\n\n\n#### **Mitigation strategies**\n\n\n| Strategy | How It Works | When to Use |\n|----------|--------------|-------------|\n| **Jittered retries** | Add random delay to retry timing | Client-side, always recommended |\n| **Exponential backoff** | Increase delay between retries | Client-side, always recommended |\n| **Request coalescing** | Combine duplicate requests | Cache layer, high fan-out |\n| **Gradual rollout** | Slowly increase traffic to recovered service | Service recovery |\n| **Circuit breakers** | Prevent calls to failing service | Dependency protection |\n| **Load shedding** | Reject excess requests gracefully | Overload protection |\n\n\n---\n\n# Geographic Traffic Patterns\n\nGlobal applications experience rolling traffic patterns as time zones cycle through peak hours. Understanding geographic distribution is critical for multi-region architecture decisions.\n\n### Follow-the-Sun Traffic\n\n\n```mermaid\nflowchart LR\n subgraph Global[\"24-Hour Global Traffic Rotation\"]\n APAC[Asia Pacific
Peak: 6-10 PM JST
UTC+9]:::primary\n EU[Europe
Peak: 6-10 PM CET
UTC+1]:::orange\n US[Americas
Peak: 6-10 PM EST
UTC-5]:::green\n end\n\n APAC -->|\"6-8 hours later\"| EU\n EU -->|\"6-8 hours later\"| US\n US -->|\"6-8 hours later\"| APAC\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\nAs the earth rotates, peak traffic moves westward. When Asia-Pacific is at peak evening hours, Europe is at work, and the Americas are asleep. Six hours later, Europe peaks while Americas wake up and Asia-Pacific sleeps.\n\n**Regional traffic distribution throughout the day**\n\n\n| Time (UTC) | Americas | Europe | Asia-Pacific | Global Total | Notes |\n|------------|----------|--------|--------------|--------------|-------|\n| 00:00 | 10% | 5% | 40% | 55% | APAC evening |\n| 06:00 | 5% | 30% | 35% | 70% | EU morning, APAC evening |\n| 12:00 | 20% | 45% | 10% | 75% | EU afternoon |\n| 18:00 | 45% | 20% | 5% | 70% | Americas afternoon |\n| 22:00 | 35% | 5% | 25% | 65% | Americas evening, APAC morning |\n\n\nFor a truly global app, there is no true \"off-peak\" time since someone is always awake. However, global peak (when the most regions overlap in active hours) is typically around 14:00-18:00 UTC when Europe is in the afternoon and Americas morning.\n\n### Architecture Implications\n\nGeographic distribution affects where you place infrastructure and how you design capacity:\n\n\n| Architecture | Capacity Strategy | Pros | Cons |\n|--------------|-------------------|------|------|\n| **Single region** | Must handle global peak | Simple, no data sync | High latency for distant users |\n| **Multi-region active-passive** | Primary sized for global, secondary for DR | Moderate complexity | Failover latency |\n| **Multi-region active-active** | Each region sized for its peak + failover | Best latency | Complex data consistency |\n| **Follow-the-sun** | Shift capacity between regions | Cost efficient | Complex orchestration |\n\n\n\n```mermaid\nflowchart TD\n subgraph SingleRegion[\"Single Region\"]\n SR[All Traffic
→ US-East]:::primary\n end\n\n subgraph MultiRegion[\"Multi-Region Active-Active\"]\n US[US Users
→ US-East]:::primary\n EU[EU Users
→ EU-West]:::orange\n AP[APAC Users
→ AP-South]:::green\n end\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\n**Capacity sizing for multi-region**\n\nEach region needs capacity for:\n\n1. Normal peak load for that region\n2. Spillover capacity if another region fails\n3. Buffer for unexpected spikes\n\nExample calculation for a 3-region setup:\n\n\n```shell\nRegion traffic at peak:\n- US-East: 40% of global at US peak\n- EU-West: 35% of global at EU peak\n- AP-South: 25% of global at APAC peak\n\nFor full failover capability:\n- Each region needs capacity for its peak + 50% of another region\n- US-East: 40% + 20% (half of EU) = 60% of global capacity\n- EU-West: 35% + 20% (half of US) = 55% of global capacity\n- AP-South: 25% + 20% (half of EU) = 45% of global capacity\n\nTotal provisioned: 160% of single-region requirement\nBenefit: Lower latency, fault tolerance, rolling peaks\n```\n\n\n### CDN and Edge Patterns\n\nContent delivery networks handle geographic distribution automatically for static content, but edge traffic patterns differ from origin patterns:\n\n\n| Traffic Type | Edge Behavior | Origin Behavior |\n|--------------|---------------|-----------------|\n| **Static content (images, JS, CSS)** | Served from edge, distributed globally | Minimal after cache warm |\n| **Dynamic content** | May pass through edge or go direct | All traffic hits origin |\n| **Personalized content** | Typically passes through to origin | Full traffic load |\n| **Live streaming** | Fan-out at edge from single origin stream | Single stream per quality level |\n| **API calls** | Edge can cache GET, others pass through | Most traffic hits origin |\n\n\nFor a typical web application with 80% static and 20% dynamic content:\n\n- CDN absorbs 80% of bandwidth\n- Origin servers handle 20% of requests but with lower bandwidth\n- Origin capacity planning focuses on dynamic request rate, not total traffic\n\n---\n\n# Analyzing Your Traffic Patterns\n\nUnderstanding your specific patterns requires systematic data collection and analysis. You cannot optimize for patterns you have not measured.\n\n### Key Metrics to Collect\n\n\n| Metric | Granularity | Retention | Purpose |\n|--------|-------------|-----------|---------|\n| **Request rate (RPS)** | Per-second | 30+ days high-res, 1 year aggregated | Primary load indicator |\n| **Concurrent users/connections** | Per-minute | 30+ days | Session-based capacity |\n| **Response latency (P50, P95, P99)** | Per-request | 7+ days detailed | Performance monitoring |\n| **Error rate** | Per-minute | 30+ days | Capacity-related failures |\n| **Resource utilization (CPU, memory)** | Per-minute | 30+ days | Infrastructure headroom |\n| **Queue depth** | Per-minute | 30+ days | Backpressure indicator |\n\n\n### Pattern Identification Process\n\n\n```mermaid\nflowchart LR\n Collect[1. Collect
Raw Metrics]:::primary --> Visualize[2. Visualize
Multiple Scales]:::orange\n Visualize --> Decompose[3. Decompose
Trend + Season]:::teal\n Decompose --> Quantify[4. Quantify
Ratios & Peaks]:::green\n Quantify --> Baseline[5. Establish
Baselines]:::purple\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n```\n\n\n#### **Step 1: Collect raw metrics**\n\nInstrument your system to capture request rates, latencies, and errors at high resolution. Use time-series databases (Prometheus, InfluxDB) that support efficient aggregation.\n\n#### **Step 2: Visualize at multiple scales**\n\nPlot traffic over different time windows to reveal patterns at each scale:\n\n- Hourly resolution for daily patterns\n- Daily resolution for weekly patterns\n- Weekly resolution for monthly patterns\n- Monthly resolution for seasonal patterns\n\n#### **Step 3: Decompose into components**\n\nBreak traffic into trend, seasonality, and residual:\n\n\n```shell\nTraffic = Trend + Seasonality + Residual\n\nTrend: Long-term growth or decline (10% monthly growth)\nSeasonality: Repeating patterns (daily cycle, weekly cycle, yearly cycle)\nResidual: Random variation and anomalies (one-time events, noise)\n```\n\n\nTools like Facebook Prophet or statsmodels can automatically decompose time series.\n\n#### **Step 4: Quantify key ratios**\n\nCalculate the ratios that drive capacity decisions:\n\n\n| Ratio | Calculation | Capacity Impact |\n|-------|-------------|-----------------|\n| **Peak-to-average (daily)** | Max(hour) / Average(day) | Minimum headroom needed |\n| **Peak-to-trough** | Max(hour) / Min(hour) | Auto-scaling range |\n| **Ramp rate** | (Peak - Pre-peak) / Time | How fast to scale up |\n| **Weekday-to-weekend** | Avg(weekday) / Avg(weekend) | Weekly scaling pattern |\n\n\n#### **Step 5: Establish baselines**\n\nCreate expected ranges for each time period:\n\n\n```shell\nExpected traffic at Tuesday 3 PM:\n- Historical average: 10,000 RPS\n- Standard deviation: 1,500 RPS\n- Normal range (2σ): 7,000 - 13,000 RPS\n- Warning threshold: < 5,500 or > 14,500 RPS\n- Critical threshold: < 4,000 or > 16,000 RPS\n```\n\n\n### Anomaly Detection\n\nWith baselines established, detect anomalies that indicate problems or opportunities:\n\n\n| Anomaly Type | Indication | Response |\n|--------------|------------|----------|\n| **Traffic significantly below baseline** | Outage, client issues, tracking bug | Investigate immediately |\n| **Traffic significantly above baseline** | Viral event, attack, or new traffic source | Scale up, investigate source |\n| **Pattern shift** | Changed user behavior, new feature | Update models |\n| **Latency spike without traffic increase** | Performance regression, resource issue | Debug, rollback if needed |\n| **Error spike** | Capacity issue, bug, dependency failure | Immediate investigation |\n\n\n---\n\n# Handling Traffic Spikes\n\nTraffic spikes are rapid increases that exceed normal patterns. They require different strategies depending on predictability, magnitude, and duration.\n\n### Spike Categories and Strategies\n\n\n```mermaid\nflowchart TD\n Spike[Traffic Spike]:::primary\n\n Spike --> Planned[Planned Spike
Known in advance]:::green\n Spike --> Recurring[Recurring Spike
Predictable pattern]:::orange\n Spike --> Organic[Organic Spike
Viral/News]:::red\n Spike --> Attack[Attack/Abuse
Malicious]:::red\n\n Planned --> PreScale[Pre-scale hours before
Load test in advance]:::green\n Recurring --> AutoScale[Scheduled auto-scaling
Pattern-based triggers]:::orange\n Organic --> ReactiveScale[Reactive scaling
+ Rate limiting]:::red\n Attack --> WAF[WAF + Rate limiting
+ Load shedding]:::red\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n\n| Category | Predictability | Response Time | Primary Strategy |\n|----------|---------------|---------------|------------------|\n| **Planned events** | High | Days/weeks | Pre-scale, load test, prepare runbooks |\n| **Recurring peaks** | High | Hours | Auto-scale with schedule, pre-warm caches |\n| **Organic viral** | Low | Minutes | Reactive scaling + graceful degradation |\n| **Attack/abuse** | Low | Seconds | Rate limiting + WAF + load shedding |\n\n\n### Pre-Scaling for Known Events\n\nWhen you know a spike is coming (product launch, marketing campaign, scheduled event), prepare in advance:\n\n\n```mermaid\nflowchart LR\n T1[T-2 weeks
Load test]:::purple --> T2[T-1 week
Fix issues]:::purple\n T2 --> T3[T-4 hours
Begin scale-up]:::orange\n T3 --> T4[T-2 hours
Warm caches]:::orange\n T4 --> T5[T-0
Event starts]:::red\n T5 --> T6[T+4 hours
Monitor closely]:::red\n T6 --> T7[T+24 hours
Gradual scale-down]:::green\n\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\n#### **Pre-scaling checklist**\n\n\n| Timing | Action | Purpose |\n|--------|--------|---------|\n| **T-2 weeks** | Load test at 1.5x expected peak | Identify bottlenecks |\n| **T-1 week** | Fix issues found in load test | Remove constraints |\n| **T-3 days** | Final load test at expected peak | Validate fixes |\n| **T-4 hours** | Begin scaling infrastructure | Allow provisioning time |\n| **T-2 hours** | Warm caches with expected hot data | Avoid cold start misses |\n| **T-1 hour** | Disable non-critical batch jobs | Free resources |\n| **T-30 min** | Alert team, open war room | Rapid response ready |\n| **During event** | Monitor dashboards, respond to alerts | Catch issues early |\n| **T+2 hours** | Begin gradual scale-down | Avoid premature reduction |\n| **T+24 hours** | Complete scale-down, post-mortem | Learn for next time |\n\n\n### Reactive Scaling Strategies\n\nFor unexpected spikes, reactive measures protect the system while scaling catches up.\n\n#### **Auto-scaling triggers**\n\n\n| Trigger Metric | Threshold Example | Scaling Action | Considerations |\n|----------------|-------------------|----------------|----------------|\n| **CPU utilization** | > 70% for 2 min | Add 2 instances | May be slow to respond |\n| **Request queue depth** | > 100 for 1 min | Add 1 instance | Early warning |\n| **Response latency P95** | > 500ms for 2 min | Add 2 instances | User-impact focused |\n| **Error rate** | > 1% for 1 min | Add 3 instances | Problem already visible |\n| **Custom: requests per instance** | > 1000 RPS per instance | Add instances | Direct load measure |\n\n\n#### **Load shedding strategies**\n\nWhen capacity is exhausted faster than you can scale, selectively reject requests to protect core functionality:\n\n\n```mermaid\nflowchart TD\n Traffic[Incoming Traffic]:::primary\n\n Traffic --> Check{At Capacity?}\n\n Check -->|No| Serve[Serve All Requests]:::green\n Check -->|Yes| Shed[Load Shedding]:::orange\n\n Shed --> Priority{Request Priority?}\n\n Priority -->|Critical| ServeCritical[Serve]:::green\n Priority -->|Normal| MaybeServe[Serve if capacity]:::orange\n Priority -->|Low| Reject[Reject gracefully]:::red\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n\n| Strategy | How It Works | Best For |\n|----------|--------------|----------|\n| **Random rejection** | Reject X% of requests randomly | Even load reduction, simple |\n| **Priority-based** | Reject low-priority requests first | Protect important users/features |\n| **Oldest-first** | Reject requests waiting longest | Prevent timeout pile-up |\n| **New-connection limiting** | Accept existing sessions, reject new | Protect active users |\n| **Geographic** | Reject or redirect distant regions | Focus on primary market |\n\n\n#### **Graceful degradation**\n\nReduce functionality to maintain core service:\n\n\n| Degradation Level | What to Disable | User Impact |\n|-------------------|-----------------|-------------|\n| **Level 1: Cosmetic** | Analytics, tracking, logging | None visible |\n| **Level 2: Enhancement** | Recommendations, personalization | Reduced experience |\n| **Level 3: Features** | Search suggestions, auto-complete | Feature reduction |\n| **Level 4: Secondary** | Comments, reviews, social features | Core function only |\n| **Level 5: Emergency** | Everything except core transaction | Minimal viable service |\n\n\n### Multi-Layer Protection\n\nDefense in depth against traffic spikes uses multiple protection layers:\n\n\n```mermaid\nflowchart TD\n Traffic[Incoming Traffic
10x spike]:::primary\n\n Traffic --> CDN[CDN Layer
Cache + WAF + Rate Limit]:::orange\n CDN -->|\"Absorbs 60%\"| LB[Load Balancer
Distribution + Health Checks]:::teal\n LB -->|\"Passes 40%\"| App[Application
Circuit Breakers + Bulkheads]:::green\n App -->|\"Throttled to sustainable\"| Cache[Cache Layer
90% hit rate]:::purple\n Cache -->|\"10% passes\"| DB[Database
Connection Pools]:::red\n\n CDN -.->|\"Rejected: 20%\"| Rejected1[Rejected]\n LB -.->|\"Rejected: 10%\"| Rejected2[Rejected]\n App -.->|\"Degraded: 5%\"| Degraded[Degraded Response]\n\n classDef primary fill:#00ceff,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n\n| Layer | Protection Mechanism | What It Absorbs |\n|-------|---------------------|-----------------|\n| **CDN/Edge** | Cache, WAF, rate limiting, DDoS protection | Static content, attacks, geographic distribution |\n| **Load balancer** | Connection limits, health checks, routing | Failed instances, connection floods |\n| **Application** | Circuit breakers, bulkheads, timeouts | Dependency failures, slow calls |\n| **Cache** | Response caching, request coalescing | Repeated reads, database protection |\n| **Database** | Connection pools, query timeouts | Connection exhaustion, long queries |\n\n\n---\n\n# Pattern-Based Architecture Decisions\n\nTraffic patterns should directly influence architectural choices. The right architecture for steady traffic differs from the right architecture for bursty traffic.\n\n### Queue-Based Buffering\n\nFor bursty traffic patterns, queues smooth out spikes by decoupling intake from processing:\n\n\n```mermaid\nflowchart LR\n Spike[Traffic Spike
1000 RPS burst]:::red --> Queue[Message Queue
Buffers excess]:::orange --> Workers[Worker Pool
100 RPS steady]:::green\n\n Queue -.->|\"Queue depth grows
during spike\"| Note[Processed over time
at sustainable rate]\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef green fill:#69db7c,stroke:#000,color:#000\n```\n\n\n\n| Traffic Pattern | Queue Benefit | Trade-off |\n|-----------------|---------------|-----------|\n| **Unpredictable spikes** | Absorb bursts without scaling | Increased latency during spikes |\n| **Processing-heavy work** | Decouple intake from processing | Queue storage costs |\n| **Downstream rate limits** | Buffer when downstream cannot scale | Potential data loss if queue overflows |\n| **Cost optimization** | Process off-peak at lower rates | Delayed processing |\n\n\n**When queues work well:**\n\n- Async processing (email, notifications, analytics)\n- Write-heavy workloads (logging, event ingestion)\n- Background jobs (report generation, data processing)\n\n**When queues do not help:**\n\n- Synchronous user-facing requests (need immediate response)\n- Read-heavy workloads (need data immediately)\n- Real-time requirements (gaming, trading)\n\n### Caching Strategies by Pattern\n\nDifferent traffic patterns call for different caching strategies:\n\n\n```mermaid\nflowchart TD\n subgraph Patterns[\"Traffic Pattern\"]\n HotSpot[Hot Spot
Few items very popular]:::red\n LongTail[Long Tail
Many items, even access]:::orange\n TimeBased[Time-Based
Content ages quickly]:::teal\n UserSpec[User-Specific
Personalized content]:::purple\n end\n\n HotSpot --> HotCache[Small cache
Long TTL
High hit rate]:::red\n LongTail --> LongCache[Large cache
Medium TTL
Moderate hit rate]:::orange\n TimeBased --> TimeCache[Any size
Short TTL
Frequent refresh]:::teal\n UserSpec --> UserCache[Partitioned cache
Per-user TTL
Session-based]:::purple\n\n classDef red fill:#ff8787,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef teal fill:#38d9a9,stroke:#000,color:#000\n classDef purple fill:#9775fa,stroke:#000,color:#000\n```\n\n\n\n| Traffic Pattern | Caching Strategy | Cache Size | TTL |\n|-----------------|------------------|------------|-----|\n| **Hot spot** (few items popular) | Small cache, long TTL | 10% of data | Hours to days |\n| **Long tail** (even distribution) | Large cache, shorter TTL | 50%+ of data | Minutes to hours |\n| **Time-based** (news, feeds) | Short TTL, aggressive refresh | Medium | Seconds to minutes |\n| **User-specific** | Per-user cache partitions | Varies | Session-based |\n| **Seasonal** | Pre-warm before season | Large during peak | Varies |\n\n\n### Read Replica Strategies\n\nFor read-heavy patterns with daily or weekly cycles:\n\n\n| Traffic Pattern | Replica Strategy | Scaling Approach |\n|-----------------|------------------|------------------|\n| **Even throughout day** | Fixed replica count | Static provisioning |\n| **Strong daily peak** | Add replicas before peak | Time-based auto-scaling |\n| **Geographic concentration** | Replicas in user regions | Geographic distribution |\n| **Read type variation** | Specialize replicas | Analytics vs OLTP separation |\n\n\n---\n\n# Traffic Forecasting for Capacity\n\nUsing historical patterns to predict future traffic enables proactive capacity management.\n\n### Forecasting at Different Horizons\n\n\n```mermaid\nflowchart LR\n subgraph Horizons[\"Forecast Horizons\"]\n Short[Short-Term
Hours to Days]:::green\n Medium[Medium-Term
Weeks to Months]:::orange\n Long[Long-Term
Quarters to Years]:::red\n end\n\n Short --> ShortUse[Auto-scaling
Pre-scaling events]:::green\n Medium --> MediumUse[Capacity procurement
Budget planning]:::orange\n Long --> LongUse[Strategic planning
Infrastructure investments]:::red\n\n classDef green fill:#69db7c,stroke:#000,color:#000\n classDef orange fill:#ffa94d,stroke:#000,color:#000\n classDef red fill:#ff8787,stroke:#000,color:#000\n```\n\n\n**Short-term forecasting (hours to days):**\n\n\n```shell\nMethod: Same time, recent period\nForecast(Tuesday 3 PM) = Average(last 4 Tuesday 3 PMs) × recent_trend\n\nExample:\n- Last 4 Tuesday 3 PMs: 9,500, 10,000, 10,200, 10,500 RPS\n- Average: 10,050 RPS\n- Recent trend: +3% week-over-week\n- Forecast: 10,050 × 1.03 = 10,350 RPS\n- Range: 9,300 - 11,400 RPS (±10% for uncertainty)\n```\n\n\nBest for: Auto-scaling decisions, event pre-scaling, staffing\n\n**Medium-term forecasting (weeks to months):**\n\n\n```shell\nMethod: Trend + seasonality\nForecast(March) = February_baseline × MoM_growth × seasonal_factor(March)\n\nExample:\n- February baseline: 1M requests/day\n- MoM growth: 8%\n- March seasonal factor: 1.05 (5% higher than February historically)\n- Forecast: 1M × 1.08 × 1.05 = 1.13M requests/day\n```\n\n\nBest for: Capacity procurement, budget planning, hiring\n\n**Long-term forecasting (quarters to years):**\n\n\n```shell\nMethod: Business projections × historical patterns\nForecast(Next Year Black Friday) =\n This Year Black Friday × Expected_Annual_Growth × Market_Adjustment\n\nExample:\n- This year Black Friday: 500K concurrent users\n- Expected annual growth: 40%\n- Market adjustment: 1.0 (no major changes expected)\n- Forecast: 500K × 1.4 × 1.0 = 700K concurrent users\n- Plan for: 840K (700K + 20% buffer)\n```\n\n\nBest for: Strategic planning, major infrastructure investments, contract negotiations\n\n### Forecast Accuracy and Validation\n\nForecast accuracy decreases with horizon length. Track and validate forecasts to improve models:\n\n\n| Horizon | Typical Accuracy | Validation Frequency |\n|---------|------------------|---------------------|\n| Hours | ±10-15% | Daily comparison |\n| Days | ±15-25% | Weekly review |\n| Weeks | ±20-30% | Monthly review |\n| Months | ±30-50% | Quarterly review |\n| Years | ±50%+ | Annual review |\n\n\n**Validation process:**\n\n\n| Method | How | When |\n|--------|-----|------|\n| **Backtesting** | Apply model to historical data, compare to actuals | Before deploying new model |\n| **Actual vs forecast** | Compare predictions to reality | After each period |\n| **Confidence intervals** | Track if actuals fall within predicted bounds | Ongoing |\n| **Error analysis** | Investigate large misses | When forecast error > 2x expected |\n| **Model refresh** | Update models with recent data | Monthly or after major changes |\n\n\n---\n\n# Summary\n\nTraffic patterns describe how load distributes over time. Understanding them is essential for efficient capacity planning and robust system design. Here are the key takeaways:\n\n\n| Concept | Key Point |\n|---------|-----------|\n| **Daily patterns** | Most apps have 3-10x variation between peak and trough. Peak is typically evening for consumer apps, business hours for B2B. |\n| **Weekly patterns** | B2B apps are weekday-heavy (5-10x weekend), consumer apps often weekend-heavy (2-3x weekday). |\n| **Seasonal patterns** | E-commerce sees 2-3x normal during holidays. Requires months of advance planning. |\n| **Event-driven** | Launches and viral moments can cause 10-100x spikes. Pre-scale when known, protect with rate limiting when not. |\n| **Geographic** | Global apps have rolling peaks across time zones. Multi-region architecture smooths this and improves latency. |\n| **Thundering herd** | Recovery from outages or cache expiry can cause sudden spikes. Mitigate with jitter and gradual restoration. |\n| **Pattern analysis** | Collect data, identify patterns at multiple scales, establish baselines, detect anomalies. |\n\n\n#### **Key ratios to remember**\n\n\n| Metric | Typical Range | Planning Implication |\n|--------|---------------|----------------------|\n| Daily peak-to-average | 2-5x | Minimum auto-scaling range |\n| Daily peak-to-trough | 3-10x | Maximum scaling range |\n| Weekday-to-weekend (B2B) | 3-5x | Weekly scaling pattern |\n| Seasonal peak (e-commerce) | 2-3x | Quarterly capacity planning |\n| Event spikes (launches) | 5-50x | Pre-scaling requirements |\n| Viral spikes (unplanned) | 10-100x+ | Protection mechanisms needed |\n\n\nTraffic patterns tell you when load arrives. But manually adjusting capacity based on patterns is error-prone and slow. The next chapter covers auto-scaling: systems that automatically adjust capacity based on real-time load and predicted patterns, turning pattern knowledge into automated action.\n\n---\n\n### References\n\n- [Netflix Capacity Planning at Scale](undefined)\n- [Google SRE Book - Handling Overload](undefined)\n- [AWS Well-Architected Framework - Performance Efficiency](undefined)\n- [Alibaba Singles' Day Architecture](undefined)","pageType":"system-design"}

Traffic Patterns

Ashish Pratap Singh

Get Premium