We, as engineers, all know that the coding agents has become an important part of our lives and we use it everyday. Have you ever paid attention to the fact that an agentic coding session, at some point, is mostly re-reading? The chat log already holds the system prompt, the repository, and a few dozen tool outputs. A short message arrives (which is essentially the user's prompt), the model reads all of it, writes a few hundred tokens, calls a tool, and the tool's output comes back as the next short message. And just repeat. Contexts well past a hundred thousand tokens are routine in this loop, and a million is the ceiling this model is built to.

Two things cost money in that loop. Reading the new tokens is compute. Remembering the old tokens between reads is storage. Both grow with the length of the conversation, and for an open-weight model, whoever runs the inference pays both bills directly. That is why the inference infrastructure half of the DeepSeek report deserves the same attention as the architecture half.

The chart below is the storage bill, per token, across four generations of DeepSeek models. Read it at a million tokens: on DeepSeek-V4.1-Flash, the conversation's global KV cache occupies about 0.89 GB of GPU memory, excluding local-window state, model weights and other runtime overhead. On DeepSeek's first model, from November 2023, the same conversation would have needed about 390 GB. That is a 437× reduction in under three years, about 4× of it in this release, and it came with a model that scores higher than its predecessor rather than lower.

To see how, it helps to have a picture of what a server holds per token and where it holds it.

What an inference engine holds

Let's start with a standard GPT-style transformer. The transformer generates one token at a time (i.e. autoregressive), and to produce each new token, every layer's attention looks back at every earlier token. What it looks at is not the earlier tokens themselves but two vectors the layer computed for each of them, a key and a value. Without the cache, generating each new token would mean rerunning the prefix through the network, so the inference engine keeps them. Caching avoids that redundant prefix computation, though attention still reads the previous keys and values. That store is the KV cache: for each layer, for each token, one key and one value.

VictoriaMetrics is a time series database (TSDB) written in Go. It's Prometheus-compatible out of the box, it speaks PromQL (with extensions via MetricsQL), accepts remote write natively, and ships as a single binary you run with CLI flags. You can deploy it as one node, and when you outgrow that, scale to a cluster without switching binaries or running a migration.

Why Is It So Much Cheaper?

There are a few concrete technical reasons the numbers look the way they do.

Custom Compression

VictoriaMetrics uses its own compression scheme for time series data. Across the public case studies, the average datapoint size on disk is consistently between 0.4 and 1.75 bytes. ARNES (the Slovenian research network) reports 0.4 bytes per datapoint. Adsterra reports 0.75 bytes. For context, a raw 64-bit float timestamp plus a 64-bit float value is 16 bytes.

The New York Times: One Topic for All Content Since 1850

The New York Times uses Kafka end-to-end to publish its content. Without any extra database in the system, all articles going back to the 1850s are stored chronologically in a single topic.

Materialized views and KTables are exposed using Kafka Streams. Because there are different schemas, different teams, and different data formats, they didn't build any separate API or microservice layer just for publishing. All services consume their data and assets directly from Kafka; what they do with that data afterwards and where they store it is up to them.

When a new article or piece of content is published, it's pushed to that same topic in Kafka and becomes accessible to every service at the same time.

Figure 1: The new New York Times log/Kafka-based publishing architecture.