Live Public repo Last commit 4 days ago

serving-cost

Serving cost math for self-hosted LLM inference: per-GPU throughput normalization, dollars per million tokens, parity premiums, cold-start budgeting.

serving-cost's week on Fridays, when it has three or more things in it. Double opt-in, one-click unsubscribe. Or follow everything James Verlander builds

Commits 2
30 days 90 days 12 months All time
Pick dates
0.5
1
Nov 18 Nov 20 Today

What's new

No entries tagged serving-cost in the last 30 days.

Recent commits

From the GitHub sync, as written
Nov 18
2 commits
Add allreduce_tax.py: tensor-parallel all-reduce wire math (message size, per-step payload, link time)
serving-cost: per-GPU, per-million-token, parity, and cold-start math

Merges and dependency updates stay hidden. All commits on GitHub

Give your projects a page like this one.

It fills itself from GitHub.