Live
Public repo
Last commit 4 days ago
serving-cost
Serving cost math for self-hosted LLM inference: per-GPU throughput normalization, dollars per million tokens, parity premiums, cold-start budgeting.
serving-cost's week on Fridays, when it has three or more things in it. Double opt-in, one-click unsubscribe. Or follow everything James Verlander builds
What's new
No entries tagged serving-cost in the last 30 days.
Recent commits
From the GitHub sync, as writtenNov 18
2 commits
Add allreduce_tax.py: tensor-parallel all-reduce wire math (message size, per-step payload, link time)
serving-cost: per-GPU, per-million-token, parity, and cold-start math
Merges and dependency updates stay hidden. All commits on GitHub
Give your projects a page like this one.
It fills itself from GitHub.