02 / UNDERSTAND
Understand language models through concepts, papers and experiments.
START HERE
Derive K/V reuse from causal attention, calculate cache memory, and separate inference caching from prefix reuse and chat history.
From a two-row hand calculation to a two-layer numeric experiment: causal masks, softmax and future-information leakage.
Compare prefix recomputation, token decoding and chunk caching with output errors, projected-row counts and actual array payloads.
Separate absolute positions, cache slots and retained representations with a two-layer sliding-attention experiment.
Three seeds and six conditions compare unchecked and keyed prefix reuse; numerical equality does not establish permission to share across scopes.
27 fixed conditions and 81 raw arrays test longest-prefix cropping, continuation positions, sliding-window fallback and scope isolation.
324 requests across cold-start traces and 648 arrays test byte-bounded prefix checkpoints, working sets, isolation and LRU eviction.
4,860 measured queries compare three admission policies, connecting byte budgets, working sets, computation and local elapsed time.
Six synthetic cases, four context conditions and 29 recorded requests. A Java experiment that separates retrieval results from HTTP 429 failures.
A Java 8 paired-block protocol with 24 live Agnes 3.0 requests, independently audited scores and reported token usage, including the limits of a perfect observed score.
Remove wording and identifier shortcuts, freeze 128 similar-distractor requests, and verify the materials with Java 8 and an independent audit. Offline preparation is not a new model result.
Profile 909 queries across signatures, lookup, copying and continuation, including the remaining slowdown.
Derive online softmax from a failing block-average baseline and separate smaller temporaries from measured speed.
Compare tensor and row-wise int8 quantization, including scale bytes and attention error under outliers.
162 archived INT8 comparisons examine tensor, token and K-channel/V-token scales, including cases where the hybrid loses.
Store real nibbles and audit group-scale overhead, padding bytes and attention output error.
Compare early-focused and recent-focused attention to explain unequal gains from the same float32 window.
Separate cache payload, work arrays and observed runtime, retaining 810 timings including slower block decoding.
PROJECTS
Reproducible Java conversation, retry and persistence boundaries, alongside attention, cache-cost and quantization-error experiments.
Can the model retrieve the same fact at different positions?
From a two-row hand calculation to a two-layer numeric experiment: causal masks, softmax and future-information leakage.
Three seeds and six conditions compare unchecked and keyed prefix reuse; numerical equality does not establish permission to share across scopes.
27 fixed conditions and 81 raw arrays test longest-prefix cropping, continuation positions, sliding-window fallback and scope isolation.
324 requests across cold-start traces and 648 arrays test byte-bounded prefix checkpoints, working sets, isolation and LRU eviction.
4,860 measured queries compare three admission policies, connecting byte budgets, working sets, computation and local elapsed time.
Profile 909 queries across signatures, lookup, copying and continuation, including the remaining slowdown.
Derive online softmax from a failing block-average baseline and separate smaller temporaries from measured speed.
Compare tensor and row-wise int8 quantization, including scale bytes and attention error under outliers.
162 archived INT8 comparisons examine tensor, token and K-channel/V-token scales, including cases where the hybrid loses.
144 comparisons and 810 timings. Recent float32 windows help recent-focused attention more; block decoding reduces work arrays but is slower on this CPU run.
Published updates from external sources.
As large language models (LLMs) become more widespread in Africa, their safety in low-resource languages remains largely untested since most red-teaming work focuses on English.…
Original sourceLarge Language Models (LLMs) rely on Multi-Head Attention (MHA) for sequence modeling, but scaling to long contexts is constrained by the quadratic cost of attention, motivating…
Original sourceOpenRouter 发布教程，讲解如何把 PDF 以文件形式连同严格 JSON schema 发给能读文件的模型，让表格以带类型的行返回，括号亏损、千分位和脚注标记等清理都写进字段描述。
Original sourceThese are future topics, not completed experiments.
Expand to explore in your browser. Link to this exercise ↗
