How this demo works

From plain English to the math. Starts simple, gets deep. Designed so anyone — a finance lead, an infrastructure VP, or an engineer — can follow the story and verify the numbers.

1. Start here — what's "goodput" and why should I care?

The kitchen story

Imagine a very expensive chef. The chef costs $1,000 an hour. The chef can cook fast, but only if the ingredients show up on time. If the delivery truck is late, the chef just stands there with a knife. You still pay the chef.

That's what's happening to GPUs every day. An H100 GPU rents for about $3 to $4 per hour. A cluster of 256 H100s costs more than $1,000 per hour to run. AI companies pay that whether the GPUs are training a model or sitting there waiting for the next batch of data.

Throughput vs goodput

Two words that sound the same but mean different things:

• Throughput — how fast the truck CAN drive. The top speed.
• Goodput — how many packages actually arrive on time at the right house. The useful work.

A truck with a 200 mph top speed and zero packages delivered has high throughput and zero goodput. A truck driving 60 mph and delivering every package on schedule has lower throughput and perfect goodput. Goodput is what you actually get paid for.

For an AI training cluster, goodput is the percentage of GPU time spent actually computing, not waiting on storage. A "95% goodput" cluster means 5% of the time the GPUs are idle — waiting for the next shuffle of training data, or waiting for a checkpoint to finish flushing to disk.

Why a few percentage points matter so much

That's the whole idea in one paragraph: storage that keeps GPUs busy converts idle GPU dollars into billable training dollars. The faster and steadier the storage, the more of the GPU bill is doing useful work.

When does goodput actually start to matter?

Two conditions have to be true at the same time. If either one isn't, the goodput math reduces to zero (and Standard B2 is the right answer).

Below both thresholds — Fine-tune lab · 1 PB territory in the demo — Standard B2 is the right answer and the goodput tiles correctly read $0. The tool will recommend Standard when you click "Auto-select optimal tier" in this regime.

Above both thresholds — Mid-scale tenant · 5 PB and up — the goodput story dominates. The Net ROI math turns positive for an Overdrive tier sized to the cluster's demand.

2. The opportunity — who has this problem

The four customer segments this is built for

What they all share

• A flash bill they're trying to shrink (drive cost + power + cooling + ops)
• A GPU bill they're trying to monetize harder (sell more billable hours from the same cluster)
• A CFO asking "why are we paying for storage we're not actively reading?"
• Customers asking "why does my training run stall for two minutes every hour at every checkpoint?"

3. What this tool shows

The page has four big numbers across the top — the "hero tiles." Each is one piece of the goodput story.

Why these four, and not a simple storage cost comparison?

Storage is a small fraction of any AI training bill. Even cutting it 80% doesn't move the conversation. GPU spend is 10–100× the storage spend, which means a 1% goodput improvement is worth more than a 50% storage discount. This tool surfaces that math directly instead of burying it inside a per-TB rate sheet.

How to read the baseline

When the page loads on Standard B2 (50 Gbps), every goodput tile shows $0 with an explainer ("Standard tier — no goodput delta to claim"). That's intentional, not a bug. Standard B2 IS the baseline — there's no delta to compare against itself. Switch to an Overdrive tier (100, 200, 300, 400, or 800 Gbps) to see the math kick in and the dollar values change.

4. Demo Walkthrough

The fastest way to learn what the tool does is to click through it. The walkthrough below takes about five minutes.

1. Open the page. Mid-scale tenant · 5 PB (128 GPUs, 70B model) is the default scenario. The hero tiles show roughly $170K/mo total monthly impact on its recommended 100 Gbps Overdrive tier.
2. Click the Standard B2 tier button. Every Overdrive-dependent tile drops to $0. This is the baseline — Standard B2 is the comparison point and has no "Overdrive delta" by definition.
3. Click "⚡ Auto-select optimal tier." The tool picks the feed floor — the smallest tier that fully streams the dataset on schedule (where Net ROI peaks) — and shows the reasoning in the live event stream below.
4. Read the recommendation. The B2 tier strip marks the recommended tier as a combined ★⚡ — best value and keeps-training-fed land on the same tier (the feed floor, the smallest tier that fully streams the dataset). It's selected by default. A higher tier buys marginally faster checkpoints/rehydration for more premium; it's a manual click, not the default.
5. Switch to Fine-tune lab · 1 PB (32 GPUs). The optimizer recommends Standard B2, not Overdrive. At this scale the math doesn't justify the Overdrive premium — and the tool says so honestly.
6. Switch to Frontier multimodal · 60 PB (768 GPUs, 405B model). Flash capacity reclaimed jumps to about 14.4 PB on the recommended 800 Gbps tier — the capacity-reclaim story for the largest training accounts.
7. Project the fleet. Set "Tenants like this (× N)" in Flash economics — the per-tenant monthly value multiplies into a total fleet opportunity (e.g. 20 mid-scale tenants → ~$2.6M/mo). Default N = 1 is the single mapped customer.
8. Click "Run." The full 8-stage pipeline executes against the demo bucket. Hero tiles fill in progressively as each stage completes; the "Open in B2 console" link on each finished stage card surfaces the actual object in the bucket so you can verify nothing is simulated. Prefer to skip the wait? Click the small "skip" link next to the Run button — tiles populate from the math immediately.

Tuning the scenario to your workload

Open the Configure panel to adjust every input directly — cluster size, GPU $/hr, checkpoint cadence, hot working set %, flash $/TB-mo, and I/O wait %. The hero tiles and Tier Comparison panel update live as you change any field. The Tier Comparison panel breaks every tier down (goodput / stall / flash / premium / net ROI) so you can compare them side-by-side.

Switching to Full pipeline mode

The Run button executes all 8 pipeline stages end-to-end in sequence: Data Lake → Training Prep → Checkpoints → Model Registry → RAG Knowledge → KV Cache → Inference → Exhaust. The first 5 stages carry the goodput / stall / flash-displacement headline economics; the last 3 (RAG, KV Cache, Exhaust) demonstrate the full set of B2 patterns in a production AI pipeline. Individual stage cards also have their own Run / Re-run buttons if you want to demo a single pattern in isolation.

5. Every field, explained — and what we left out

Fields included

Workload profile

Flash cost model

Goodput scenario

Fields intentionally not exposed as inputs

6. The math — formulas and derivation

Every formula below is what the tool computes. Each is presented with a worked example using the Large tenant cluster · 25 PB profile (512 GPUs, 70B, 400 Gbps — the tier the math recommends) so the chain of reasoning is reproducible by hand.

6.1 — Goodput Recovered / mo

prefetch_demand C     = dataset_TB × 1000 × epochs × 8 / (run_days × 86400)   // Gbps, sustained
streamed_fraction     = cold (uncached) % of the dataset, streamed from B2 each pass
starvation(g)         = streamed_fraction × max(0, 1 − g / C)                 // GPU idle fraction
training_hrs_per_mo   = 720 × cluster_utilization                            // default 80% → 576 hrs
goodput_$_per_mo      = cluster_$_per_hr × (starvation(standard) − starvation(tier)) × training_hrs_per_mo

What it measures — keeping the GPUs fed

The dollar value of GPU time recovered by keeping the cluster fed with sharded training data. The cold/uncached portion of the dataset is streamed live from B2 during the run. Prefetching and sharding hide latency, not a bandwidth shortfall — so when a tier can't sustain the prefetch demand, the prefetch buffer drains and the GPUs starve waiting on cold reads. This is the read path — independent of checkpoint mode (async checkpointing does nothing to it). It's the core reason Overdrive matters for datasets too big to cache. Grounded in NVIDIA DGX SuperPOD storage targets (~4 GB/s/GPU read) and the tf.data / PyTorch fact that prefetch is latency-hiding only — verified 2026-06-06 (NVIDIA, tf.data).

Worked example (Large tenant cluster · 25 PB on 400 Gbps Overdrive, 70% streamed)

prefetch_demand C   = 25,000 × 1000 × 2 × 8 / (14 × 86400) ≈ 331 Gbps
starvation(50)      = 0.70 × max(0, 1 − 50/331)  = 0.594   // Standard idles ~59%
starvation(400)     = 0.70 × max(0, 1 − 400/331) = 0       // 400 fully feeds the demand
cluster_$/hr        = 512 × $4 = $2,048
training_hrs_per_mo = 720 × 0.80 = 576
goodput             = 2,048 × (0.594 − 0) × 576
                    ≈ $700,900 / mo

6.2 — Checkpoint Stall Eliminated

stall_std_min          = (ckpt_GB × 8) / standard_gbps / 60
stall_od_min           = (ckpt_GB × 8) / tier_gbps     / 60
delta_min              = max(0, stall_std_min − stall_od_min)
ckpts_per_hr           = 60 / cadence_min
training_hrs_per_mo    = 720 × cluster_utilization              // default 80% → 576 hrs
hours_recovered_per_mo = (delta_min / 60) × ckpts_per_hr × training_hrs_per_mo × concurrent_jobs
stall_$_per_mo         = hours_recovered_per_mo × cluster_$_per_hr × ckpt_stall_multiplier

What it measures

Every training run flushes checkpoints periodically. Under synchronous checkpointing the cluster stalls — every GPU waits on the write. Standard B2 at 50 Gbps takes about 2.1 minutes to flush an 800 GB checkpoint. Overdrive at 400 Gbps does it in ~16 seconds. That ~1.87 minute delta, repeated hourly across 4 concurrent training jobs over the 576 effective training hours, adds up (scaled by the checkpoint-write multiplier above).

Worked example (Large tenant cluster · 25 PB on 400 Gbps)

stall_std           = (800 × 8) / 50  / 60 = 2.13 min
stall_od            = (800 × 8) / 400 / 60 = 0.27 min
delta               = 1.87 min
ckpts/hr            = 60 / 60 = 1
training_hrs_per_mo = 720 × 0.80 = 576
hours/mo            = (1.87/60) × 1 × 576 × 4 jobs
                    = 71.7 hours
stall_$/mo          = 71.7 × $2,048
                    ≈ $146,801 / mo

6.3 — Flash capacity reclaimed

dataset_PB     = pipeline_training_TB / 1,000     (decimal PB, matches spec §4)
hot_PB         = dataset_PB × hot_working_set% / 100
displaced_PB   = hot_PB × overdrive_replace% / 100 × tier_capability
displaced_TB   = displaced_PB × 1,000
flash_$        = displaced_TB × flash_$_per_TB_per_mo
b2_overdrive_$ = displaced_TB × tier_storage_rate
net_savings    = max(0, flash_$ − b2_overdrive_$)

What it measures

The amount of flash a customer no longer needs to provision because Overdrive serves the hot working set at line rate — reclaimed capacity the neocloud can resell. For the 60 PB Frontier multimodal tenant with an 18 PB hot tier (30%), Overdrive 800 serving 80% of that hot tier at 100% capability on a 768-GPU cluster reclaims about 14.4 PB. At $100/TB·mo flash and $29/TB·mo B2 Overdrive 800, that's ~$1.02M/mo of net storage savings.

Worked example (Large tenant cluster · 25 PB on 400 Gbps)

dataset       = 25,000 TB / 1,000 = 25 PB
hot           = 25 × 0.32        = 8 PB
displaced     = 8 × 0.80 × 0.78  = 5.0 PB (5,000 TB)
flash_$       = 5,000 × $100     = $500,000
b2_od_$       = 5,000 × $21      = $105,000
net           = $500,000 − $105,000
              ≈ $395,000 / mo

6.4 — New-capacity revenue / mo

freed_TB        = Σ STAGE_FLASH_FREED_TB[stage] for stage in completed_stages
margin_per_TB   = flash_sell_price − b2_storage_rate
revenue_$_per_mo = max(0, freed_TB × margin_per_TB)

What it measures

The neocloud's resale opportunity. As each demo stage offloads its data tier to B2, that flash capacity is freed up. The neocloud can resell that flash to a new GPU customer at the going rate (~$100/TB-mo) while paying B2 only the storage rate (~$6.95–$19/TB-mo depending on tier). The margin is recurring revenue.

Per-stage flash-freed assumptions

These per-stage fractions are educated estimates. Validate against actual customer deployment patterns before externalizing as authoritative numbers.

6.5 — Tier Premium and Net ROI (Tier Comparison table)

storage_TB         = training_TB + checkpts_TB + models_TB + logs_TB
standard_$_per_mo  = storage_TB × $6.95
tier_$_per_mo      = max(tier_min_monthly_commit,
                         storage_TB × tier_storage_rate + tier_network_fee)
tier_premium       = max(0, tier_$_per_mo − standard_$_per_mo)

net_ROI_per_mo     = goodput_$ + stall_$ + net_flash_$ − tier_premium

This is what the "Auto-select optimal tier" button uses to pick the best tier. The winner is whichever Overdrive tier maximizes net ROI for the configured workload. If no Overdrive tier produces positive ROI (small workloads where the tier premium exceeds the gains), it recommends Standard B2.

Two recommendations: ★ best value and ⚡ keeps training fed

The tier strip marks two tiers, because the cheapest-ROI tier and the tier that fully feeds the cluster aren't always the same:

• ★⚡ recommended — the feed floor, the smallest tier that fully feeds the cluster (what "Auto-select" picks). Best value and keeps-fed land here together. This is the default-selected pill.
• ⚡ keeps training fed — the smallest tier whose throughput covers the workload's prefetch demand: dataset_TB × 1000 × epochs × 8 ÷ (run_days × 86400) Gbps. That's the rate at which the data pipeline can stay ahead of the GPUs for the whole run.

⚡ never marks a tier below ★ — if the best-value tier already covers the feed floor, the two combine into a single ★⚡ badge ("best value & keeps training fed"). ⚡ only appears as a separate, higher tier when the best-value pick would under-feed the cluster, so it always reads as the performance up-tier.

Why this matters across the two tools: both calculators recommend the feed floor — the smallest tier that fully streams the dataset on schedule — using the same prefetch formula, so they recommend the same tier for the same workload (5 PB → 100, 10 PB → 200, 25 PB → 400, 60 PB → 800 in both). The operator and tenant frame the value differently (the operator as capacity reclaimed for resale → growth, the tenant as their own spend), but the recommended throughput tier is identical.

6.6 — The bandwidth-capability cap

prefetch_demand_gbps = dataset_TB × 1000 × epochs × 8 / (run_days × 86400)
tier_capability      = min(1, tier_gbps / prefetch_demand_gbps)

An Overdrive tier that can't sustain the prefetch demand can't fully feed the cluster. The capability cap is the fraction of that demand a tier serves — measured against the prefetch demand (dataset × epochs / run-time), the same basis as the ⚡ keeps-fed tier and the feed-driven goodput — so the Capability column, the recommendation, and the goodput all agree. It prorates Goodput Recovered and Flash Displaced.

Examples (25 PB profile, prefetch demand ≈ 331 Gbps):

• 100 Gbps tier: capability = 100/331 = 30%. Undersized — the GPUs starve on the cold reads it can't keep up with.
• 300 Gbps tier: capability = 300/331 = 91%. Nearly fully fed.
• 400 Gbps tier: capability = 100% (400 ≥ 331). Fully feeds the prefetch demand — the recommended ★⚡ feed floor for the 25 PB profile.

7. Assumptions to validate against your workload

Where the I/O wait baselines come from

The 4.0% (Standard) and 0.8% (Overdrive) defaults sit in the middle of public ranges reported by storage and ML infrastructure vendors. They are not measurements of your specific workload — they describe what well-tuned vs un-tuned training-storage stacks typically look like in the industry.

8. FAQ

Where do the 4% / 0.8% I/O wait numbers come from?

They are working defaults pending validation against telemetry from a representative training run. The "Where the I/O wait baselines come from" subsection in §7 above cites the public source ranges. To produce numbers you can budget against, take a measurement of your own cluster (GPU SM efficiency from nvidia-smi or DCGM gives the inverse of I/O wait %) and update the Configure fields accordingly.

Why do the ★ best-value and ⚡ keeps-fed tiers sometimes differ?

The recommended ★⚡ tier is the feed floor: the smallest tier that fully streams your dataset on schedule (prefetch demand). Best-value and keeps-fed are the same tier — feeding below it starves the GPUs, and paying above it buys feed you already have — so every profile shows a combined ★⚡ (5 PB → 100, 10 PB → 200, 25 PB → 400, 60 PB → 800). A higher tier is always available as a manual click for faster checkpoints/rehydration. See §6.5.

Does the GenAI tool recommend the same tier as this one?

Yes. Both calculators recommend the feed floor — the smallest tier that fully streams the dataset on schedule — using the same prefetch formula, so they recommend the same tier for the same workload (5 PB → 100, 10 PB → 200, 25 PB → 400, 60 PB → 800 in both). The operator and tenant frame the value differently (the operator as flash capacity reclaimed for resale → growth, the tenant as their own spend), but the recommended throughput tier is identical.

Why isn't there an object-storage vs object-storage comparison?

The headline math here measures GPU-productivity impact, not storage line-item savings. A 5% improvement on a multi-hundred-thousand-dollar monthly GPU bill is worth more than even a large percentage discount on a storage line item. The tool is intentionally focused on the GPU-economic story.

What if my workload's training I/O is heavier?

The bandwidth demand is the prefetch demand — dataset × epochs × 8 / run-time — derived from the editable Epochs per run and Run duration inputs. Heavier I/O shows up as shorter runs or more epochs, which raise the demand directly; raise the streamed (uncached) % too if more of the dataset must come from B2 each pass. The Tier Comparison "Capability" column shows whether a given tier can keep up with the resulting demand.

The demo only uploads ~200 MB. Why are you projecting petabyte numbers?

The live stage runs demonstrate the patterns end-to-end — multipart uploads, parallel range reads, mid-run checkpointing, model registry, inference cold-start — using small data so the demo finishes in under a minute. The dollar projections run against the configured workload (the workload profile preset). Each completed stage card explicitly says "↓ Projects to X PB freed" to flag the scale shift.

Why does the page open with Configure expanded?

So you can see — and change — every input that drives the headline numbers. Nothing is hidden behind a magic button. If you want a cleaner view, click the Configure summary to collapse it; the hero tiles stay populated.

Can you explain the Capability column in the Tier Comparison table?

It's the fraction of the cluster's read demand the tier can actually serve. Yellow (under 100%) = the tier is undersized for this cluster; goodput and flash benefits are prorated. Green (100%) = the tier saturates the workload; full benefit. The optimizer prefers the smallest 100%-capable tier because anything larger adds tier premium without adding benefit.

Why doesn't the Goodput tile change when I switch between Overdrive tiers?

It does — but it only changes when the bandwidth capability changes. Once a tier is big enough to saturate the cluster's read demand (cluster GPUs × 1 Gbps), every larger tier delivers the same goodput recovery at 100% capability; smaller tiers show lower goodput because they're undersized and prorated. This is the model behaving honestly — once a tier feeds the cluster, more bandwidth doesn't recover more goodput.

What does "Standard rates — recommend Overdrive to recover goodput" mean? Is Standard bad?

No. For small workloads Standard is the right answer (the Fine-tune lab · 1 PB profile is built around exactly that case). The note is just saying Standard is the baseline by definition, so there's no delta to recover when comparing it to itself. At Overdrive tiers the math measures the delta from the Standard baseline.

What does the Clean up button do? Is it safe?

Cleanup deletes only objects under the configured DEMO_PREFIX (default goodput-demo/). The confirmation prompt names the exact bucket and prefix being wiped. Other prefixes in the same bucket are untouched. Even so — use a dedicated demo bucket if you're running this against your own credentials.

Live demo: https://genai.backblazedemos.xyz/goodput/
Author: Kevin Lott · klott@backblaze.com