Profiling KV reuse: saved work versus added overhead

01 / QUESTION

Profile 909 queries across signatures, lookup, copying and continuation, including the remaining slowdown.

METHOD

Freeze protocol and source before execution, save raw records, then independently recompute metrics and verify hashes.

RESULT

909 queries; fresh/rehash/frozen medians are 252.7/682.7/273.6 microseconds.

OBSERVATIONS

909 queries; fresh/rehash/frozen medians are 252.7/682.7/273.6 microseconds.

CONCLUSION

Conclusions are limited to the stated variables and acceptance criteria, retaining failures and unverified scope.

LIMITATIONS

Teaching-scale fixed conditions, not online model quality, production throughput or hardware safety. The complete assistant and VL01 remain unfinished.

REPRODUCE

The article provides a pinned revision, environment, runner and independent audit commands.

1. Mechanisms

   Profiling KV reuse: saved work versus added overhead

   Profile 909 queries across signatures, lookup, copying and continuation, including the remaining slowdown.

   2026-10-079 MINPublished
2. PROJECT / 002

   Java LLM Practice

   Reproducible Java conversation, retry and persistence boundaries, alongside attention, cache-cost and quantization-error experiments.

   Building
3. Mechanisms

   LLM mechanics (7): why is caching slower even when it saves computation?

   4,860 measured queries compare three admission policies, connecting byte budgets, working sets, computation and local elapsed time.

   2026-10-0512 MINPublished