Packed KV, residual windows and block cost

01 / QUESTION

144 comparisons and 810 timings. Recent float32 windows help recent-focused attention more; block decoding reduces work arrays but is slower on this CPU run.

METHOD

Freeze the protocol, execute it, independently audit raw records, then recheck an export from the Git commit.

RESULT

144 comparisons and 810 timings. Recent float32 windows help recent-focused attention more; block decoding reduces work arrays but is slower on this CPU run.

OBSERVATIONS

144 comparisons and 810 timings. Recent float32 windows help recent-focused attention more; block decoding reduces work arrays but is slower on this CPU run.

CONCLUSION

Read the three question-led tutorials, including negative results and validation boundaries.

LIMITATIONS

Fixed synthetic arrays and CPU NumPy; no model-task quality, GPU speed, peak RSS or device-memory measurement.

REPRODUCE

Each article provides a pinned revision, fresh output directory, run and audit commands. Execute the shared protocol once.

1. Mechanisms

   Packed KV groups: what does 4-bit actually save?

   Store real nibbles and audit group-scale overhead, padding bytes and attention output error.

   2026-10-1110 MINPublished
2. Mechanisms

   KV residual windows: do recent tokens deserve higher precision?

   Compare early-focused and recent-focused attention to explain unequal gains from the same float32 window.

   2026-10-1110 MINPublished
3. Mechanisms

   Block KV decoding: less workspace, more time

   Separate cache payload, work arrays and observed runtime, retaining 810 timings including slower block decoding.

   2026-10-1110 MINPublished
4. PROJECT / 002

   Java LLM Practice

   Reproducible Java conversation, retry and persistence boundaries, alongside attention, cache-cost and quantization-error experiments.

   Building