Appendix D · Mental Models & Decision Trees

A collection of decision frameworks and mental models for CDN engineering. These are the heuristics principal engineers use to reason about CDN behavior quickly, without needing to run simulations.

Should I Cache This?

Is the response identical for all users?
├── No  → Cache-Control: private (browser only) or no-store
│         Examples: shopping cart, account dashboard, personalized feed
│
└── Yes → Can it be revalidated cheaply (ETag / Last-Modified)?
          ├── Yes → Cache with short max-age + conditional request support
          │         Examples: API responses with version IDs, database records
          │
          └── No  → How often does the content change?
                    ├── Never       → max-age=31536000, immutable
                    │                 Examples: hashed assets, video segments
                    ├── Rarely      → max-age=86400 (1 day)
                    │                 Examples: fonts, PDFs, images
                    ├── Daily       → max-age=3600 (1 hour) + stale-while-revalidate
                    │                 Examples: product listings, blog articles
                    ├── Frequently  → max-age=60 + stale-while-revalidate=300
                    │                 Examples: inventory counts, prices
                    └── Real-time   → max-age=5 (live streams) or no-store
                                      Examples: live sports scores, financial ticks

What TTL Should I Use?

The TTL tradeoff is always: freshness vs. CDN efficiency.

SWR = stale-while-revalidate. Serve stale content instantly while fetching fresh copy in the background.

Thundering Herd Decision Tree

Multiple concurrent requests for the same uncached resource?
│
└── Is this a shared cache (CDN, origin shield)?
    ├── Yes → Use singleflight.Group to collapse concurrent misses
    │         → Only ONE upstream request, all waiters receive the result
    │
    └── No  → Multiple browser tabs? Multiple microservices?
              └── Client-side: stagger requests with jitter
                  Service-side: use a distributed lock (Redis SETNX)

Stale TTL expiry causes burst at cache expiry time?
│
└── Use stale-while-revalidate:
    → Content remains "fresh" for 30s more (serve stale instantly)
    → Background fetch rehydrates the cache
    → Eliminates the expiry spike entirely
    
OR use XFetch (probabilistic early refresh):
    → Randomly start refreshing BEFORE expiry
    → No single expiry moment → no thundering herd

Cache Key Design Checklist

When adding a new query parameter or header to your application, ask:

1. Does it change the response body?
   Yes → Include it in cache key
   No → Exclude it (reduces cache fragmentation)

2. Is it unbounded (e.g., user ID, session token)?
   Yes → Never include in cache key (creates cache pollution)
   Strip it, or bypass CDN caching entirely

3. Is it a tracking parameter (utm_source, fbclid, gclid)?
   Strip it from the cache key (never affects content)
   Use Cache-Key VCL / Cloudflare transform rules / Fastly custom VCL

4. Does the response vary by Accept-Encoding?
   Yes → Add Vary: Accept-Encoding
   CDN creates separate cache entries for gzip, brotli, identity

5. Does the response vary by Accept-Language?
   Yes → Vary: Accept-Language or use URL path per language

Cache Tier Selection

Traffic volume per origin?
├── < 100 RPS    → Single CDN layer is sufficient
│
├── 100–10k RPS  → Add CDN with origin shield
│                   Edge (30s TTL) → Shield (300s TTL) → Origin
│
└── > 10k RPS   → Multi-tier CDN + distributed origin shield
                  + in-process cache at origin for hot items
                  + singleflight at every tier

Memory cache sizing rule of thumb:

Working set size (bytes) = median_item_size × unique_items_per_hour × revalidation_window

If you have 10k unique products, each averaging 10 KB, refreshed every 5 minutes:

10,000 items × 10 KB = 100 MB for the full working set

If you can only cache 50 MB, you’ll have a ~50% miss ratio in steady state (assuming uniform access). Add an LRU layer with hot-item bias.

Compression Algorithm Selection

Content type?
├── Already compressed (JPEG, PNG, video, WASM) 
│   → Skip compression (will be larger, not smaller)
│
└── Compressible (HTML, CSS, JS, JSON, SVG, plain text)
    │
    └── Client supports brotli (Accept-Encoding: br)?
        ├── Yes → Use brotli (20–30% better ratio than gzip, similar CPU)
        │
        └── No  → Client supports gzip?
                  ├── Yes → Use gzip (universal support, fast)
                  └── No  → Serve identity (uncompressed)

Response size?
├── < 1 KB → Skip compression (overhead > savings for tiny responses)
└── > 1 KB → Always compress compressible content

Origin Shield Sizing

Number of shield nodes:

If traffic_to_origin_without_shield = T requests/s
And shield_TTL = N seconds
And unique_items_requested_per_N_seconds = U

Then shield reduces origin to: U / N requests/s (one miss per item per TTL window)

Shield nodes needed = max(1, ceil(T / node_capacity))

For a single-node shield serving 10,000 edge requests/s with 300s TTL:

• If working set has 1,000 unique items: ~3.3 origin requests/s
• If working set has 100,000 unique items: ~333 origin requests/s

The shield’s cache size must comfortably hold the working set.

HTTP/3 Adoption Decision

Is your user base on mobile or high-latency networks?
├── Yes → Prioritize HTTP/3; measurable 15–30% improvement on lossy links
│
└── No  → HTTP/2 is sufficient for datacenter-quality paths

Does your CDN vendor support HTTP/3?
├── No  → Wait for vendor support before implementing
│
└── Yes → Enable HTTP/3 with Alt-Svc fallback to HTTP/2
          (no downside — clients that don't support H3 use H2 automatically)

Can you deploy QUIC? (UDP 443 not blocked at your edge)
├── No  → QUIC blocked by firewall; HTTP/3 won't work
│         Consider: if customers are on corporate networks, H3 gains are limited
│
└── Yes → Enable H3; measure adoption rate in 30 days

SLO Tier Selection

The rule: SLO should be set below what you can actually achieve. If your system is at 99.97% reliability, set SLO at 99.9%. The gap between actual and SLO is your buffer for incidents.

Setting SLO at 99.99% when you achieve 99.97% means you’re always burning error budget, which prevents the team from shipping new features.

Purge Strategy Selection

How frequently do you need to invalidate cached content?
│
├── Rarely (< 1/day)
│   → URL-based purge is sufficient (specify exact URLs to purge)
│
├── Regularly (dozens per day)
│   → Use surrogate keys / cache tags
│   → Tag responses: "article-123", "author-456"
│   → Purge by tag on content update: purge("article-123")
│
└── Constantly (100s per day, CMS-driven)
    → Surrogate keys + webhook from CMS on publish event
    → CDN purge API called by publish webhook
    → Consider short TTLs (60s) + stale-while-revalidate as alternative
    
How many URLs does a content update affect?
├── 1 URL       → URL purge
├── 10–100 URLs → Surrogate key covering those URLs
└── 1000+ URLs  → Short TTL is better than purging 1000 URLs

Security Checklist for CDN Engineers

Before any CDN deployment to production:

• Origin not reachable from internet — IP allowlist or mTLS
• Signed URLs for private content — HMAC-SHA256 with constant-time comparison
• WAF enabled — OWASP CRS at minimum
• TLS 1.2 minimum — no TLS 1.0/1.1, no SSLv3
• HSTS set — with preload for critical domains
• no-store on sensitive paths — auth callbacks, session tokens, API keys
• Purge API authenticated — not publicly accessible
• Metrics endpoint restricted — /metrics should not be public
• Timing-safe signature comparison — hmac.Equal, not ==
• Key rotation procedure documented — tested at least once
• Rate limiting on public endpoints — prevents DoS via cache miss amplification