{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"模拟问答：旧 key 还能比较吗？","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.md","pages":[{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","unit":"怎样增加候选，同时让历史表示仍可比较？","unitId":"i-dictionary-and-updates","chapter":"I","number":1}],"aliases":[{"oldId":"b-88d1ee9ab1d748201810a088","targetId":"b-88d1ee9ab1d748201810a088","page":"reading-opening","operation":"move","beforeSHA256":"50fe556e9536fd3c02223a0c9bad7aa6fca6d46df7b0ed3b12b73064e6a028fb","afterSHA256":"50fe556e9536fd3c02223a0c9bad7aa6fca6d46df7b0ed3b12b73064e6a028fb","reason":"Same source body in authored semantic page"},{"oldId":"b-13d6885b67e12d97a23788dd","targetId":"b-13d6885b67e12d97a23788dd","page":"reading-opening","operation":"move","beforeSHA256":"32e3dfb57a0c0ee8ebfaa6dba8b4550a4d63de60d198ddcb5319190026ceea8c","afterSHA256":"32e3dfb57a0c0ee8ebfaa6dba8b4550a4d63de60d198ddcb5319190026ceea8c","reason":"Same source body in authored semantic page"},{"oldId":"b-fc25a5d4ac7a3035ae158c02","targetId":"b-fc25a5d4ac7a3035ae158c02","page":"reading-opening","operation":"move","beforeSHA256":"3372b11342ebcdd9d3768ed439f5cd51b1b41364077b222a213f25f95759dd5d","afterSHA256":"3372b11342ebcdd9d3768ed439f5cd51b1b41364077b222a213f25f95759dd5d","reason":"Same source body in authored semantic page"},{"oldId":"b-e633ef67f3f545df5bc3fd73","targetId":"b-e633ef67f3f545df5bc3fd73","page":"reading-opening","operation":"move","beforeSHA256":"e3e35f8636c3b9db838cbc333a9a8d0c36f933bffd7004f49afcb2610515061e","afterSHA256":"e3e35f8636c3b9db838cbc333a9a8d0c36f933bffd7004f49afcb2610515061e","reason":"Same source body in authored semantic page"},{"oldId":"b-9fbbd2d021f757c70896106c","targetId":"b-9fbbd2d021f757c70896106c","page":"reading-opening","operation":"move","beforeSHA256":"958821f405d7e1cc1f7d17fd6eac5763b7a41dd75223b9498536cfae1b46711b","afterSHA256":"958821f405d7e1cc1f7d17fd6eac5763b7a41dd75223b9498536cfae1b46711b","reason":"Same source body in authored semantic page"},{"oldId":"b-8250f30b1a83f5ccb44cfa8d","targetId":"b-8250f30b1a83f5ccb44cfa8d","page":"reading-opening","operation":"move","beforeSHA256":"b130cb7eb53ce9451428fb245c8e3837e10b1d85e7c7ac232b00bb72532212b0","afterSHA256":"b130cb7eb53ce9451428fb245c8e3837e10b1d85e7c7ac232b00bb72532212b0","reason":"Same source body in authored semantic page"},{"oldId":"b-793fd1306cea1ca28c0de801","targetId":"b-793fd1306cea1ca28c0de801","page":"reading-opening","operation":"move","beforeSHA256":"10d873dba3a7aa2449a5b166cc444c9f37e8554d9cfdb7f3a19d36ca93521fe2","afterSHA256":"10d873dba3a7aa2449a5b166cc444c9f37e8554d9cfdb7f3a19d36ca93521fe2","reason":"Same source body in authored semantic page"},{"oldId":"b-ef8cbbc098a73ce395caa901","targetId":"b-ef8cbbc098a73ce395caa901","page":"reading-opening","operation":"move","beforeSHA256":"d0e9e795cf11b971fb65629f1b3c0570186830e14caf096b9e067af88c120880","afterSHA256":"d0e9e795cf11b971fb65629f1b3c0570186830e14caf096b9e067af88c120880","reason":"Same source body in authored semantic page"},{"oldId":"b-f4b153b977e41f4781101f7c","targetId":"b-f4b153b977e41f4781101f7c","page":"reading-opening","operation":"move","beforeSHA256":"a8d292570de077c1d170cf1e02e3f756c5e6147229fb804255ed87911e3c1daf","afterSHA256":"a8d292570de077c1d170cf1e02e3f756c5e6147229fb804255ed87911e3c1daf","reason":"Same source body in authored semantic page"},{"oldId":"b-014075331eed073b430d0909","targetId":"b-014075331eed073b430d0909","page":"reading-opening","operation":"rewrite","beforeSHA256":"e1b4aa1fca68c5e88657325aad237ce451f204fb235c9af78591522f824c2c58","afterSHA256":"86cbd257797f61ba401eafd7875a7b546be0a85564bf424a3a55bd986b32be3a","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-da0cfb6881ed0240e8b90afa","targetId":"b-da0cfb6881ed0240e8b90afa","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-88d1ee9ab1d748201810a088","type":"h1","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-88d1ee9ab1d748201810a088","content":"# 模拟问答：旧 key 还能比较吗？","sha256":"50fe556e9536fd3c02223a0c9bad7aa6fca6d46df7b0ed3b12b73064e6a028fb","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-13d6885b67e12d97a23788dd","prerequisites":[{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-13d6885b67e12d97a23788dd","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-13d6885b67e12d97a23788dd","content":"本页把前两页对象放回一条可追踪过程。纸面步骤与Notebook时序有细微差异，说明已明确标注。","sha256":"32e3dfb57a0c0ee8ebfaa6dba8b4550a4d63de60d198ddcb5319190026ceea8c","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-88d1ee9ab1d748201810a088","nextBlock":"b-fc25a5d4ac7a3035ae158c02","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-fc25a5d4ac7a3035ae158c02","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-fc25a5d4ac7a3035ae158c02","content":"以下为备课用模拟对话，不是实际课堂记录。","sha256":"3372b11342ebcdd9d3768ed439f5cd51b1b41364077b222a213f25f95759dd5d","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-13d6885b67e12d97a23788dd","nextBlock":"b-e633ef67f3f545df5bc3fd73","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-e633ef67f3f545df5bc3fd73","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-e633ef67f3f545df5bc3fd73","content":"**学生：**“你说是前面算过的特征，图上到底哪几个？”","sha256":"e3e35f8636c3b9db838cbc333a9a8d0c36f933bffd7004f49afcb2610515061e","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-fc25a5d4ac7a3035ae158c02","nextBlock":"b-9fbbd2d021f757c70896106c","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-9fbbd2d021f757c70896106c","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-9fbbd2d021f757c70896106c","content":"**教师：**“队列里的k1到k4是本步开始前就存在的。正在两视图分支里生成的k5是这次的新key。比较结束后k1离开，k5进入；你看下一批字典仍有k2、k3、k4。”","sha256":"958821f405d7e1cc1f7d17fd6eac5763b7a41dd75223b9498536cfae1b46711b","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e633ef67f3f545df5bc3fd73","nextBlock":"b-8250f30b1a83f5ccb44cfa8d","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-8250f30b1a83f5ccb44cfa8d","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-8250f30b1a83f5ccb44cfa8d","content":"**学生：**“编码器已经变了，它们为什么还能比？”","sha256":"b130cb7eb53ce9451428fb245c8e3837e10b1d85e7c7ac232b00bb72532212b0","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9fbbd2d021f757c70896106c","nextBlock":"b-793fd1306cea1ca28c0de801","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-793fd1306cea1ca28c0de801","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-793fd1306cea1ca28c0de801","content":"**教师：**“这是设计需要处理的困难。query可以随梯度更新，key参数用EMA缓慢跟随，减少相邻批之间的变化。旧向量没有重算，所以只能说缓和漂移，不能说完全一致。队列越长，最旧条目越老，取舍就更明显。”","sha256":"10d873dba3a7aa2449a5b166cc444c9f37e8554d9cfdb7f3a19d36ca93521fe2","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8250f30b1a83f5ccb44cfa8d","nextBlock":"b-ef8cbbc098a73ce395caa901","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-ef8cbbc098a73ce395caa901","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-ef8cbbc098a73ce395caa901","content":"**学生：**“那EMA是不是改了队列里面的数？”","sha256":"d0e9e795cf11b971fb65629f1b3c0570186830e14caf096b9e067af88c120880","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-793fd1306cea1ca28c0de801","nextBlock":"b-f4b153b977e41f4781101f7c","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-f4b153b977e41f4781101f7c","type":"p","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-f4b153b977e41f4781101f7c","content":"**教师：**“没有。看箭头：EMA连的是两套网络参数，FIFO操作连的是向量槽位。把这两条箭头分开，整步就清楚了。”","sha256":"a8d292570de077c1d170cf1e02e3f756c5e6147229fb804255ed87911e3c1daf","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ef8cbbc098a73ce395caa901","nextBlock":"b-014075331eed073b430d0909","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]},{"id":"b-014075331eed073b430d0909","type":"figure","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-014075331eed073b430d0909","content":"图内标签与数值：P17 学习表示机制图，状态 3；同一输入；视图 a；视图 b；query 编码器；key 编码器；q；k⁺ = k₅；L = −ln p⁺；比较用旧字典；k₁ … k₄；q 与所有候选比较；k⁺ 是本轮正确项；梯度 → θq；EMA；k₅ 入队；k₁ 退队；k₂ k₃ k₄ k₅；下一批的字典；按原文算法 1：更新 query，再平均 key 参数并维护下一批的字典。\n\n用可指认的key身份解释历史复用，不靠口头想象隐藏对象。 查看对应图解状态\n\n- 同一输入\n- 视图 a\n- 视图 b\n- query 编码器\n- key 编码器\n- q\n- k⁺ = k₅\n- L = −ln p⁺\n- 比较用旧字典\n- k₁ … k₄\n- q 与所有候选比较\n- k⁺ 是本轮正确项\n- 梯度 → θq\n- EMA\n- k₅ 入队；k₁ 退队\n- k₂ k₃ k₄ k₅\n- 下一批的字典\n- 按原文算法 1：更新 query，再平均 key 参数并维护下一批的字典。用可指认的key身份解释历史复用，不靠口头想象隐藏对象。 [查看对应图解状态](/#/l3-r2-p17/2)[查看完整图解](/#/l3-r2-p17/2)","sha256":"86cbd257797f61ba401eafd7875a7b546be0a85564bf424a3a55bd986b32be3a","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"用可指认的key身份解释历史复用，不靠口头想象隐藏对象。 查看对应图解状态","labels":["同一输入","视图 a","视图 b","query 编码器","key 编码器","q","k⁺ = k₅","L = −ln p⁺","比较用旧字典","k₁ … k₄","q 与所有候选比较","k⁺ 是本轮正确项","梯度 → θq","EMA","k₅ 入队；k₁ 退队","k₂ k₃ k₄ k₅","下一批的字典","按原文算法 1：更新 query，再平均 key 参数并维护下一批的字典。"],"description":"用可指认的key身份解释历史复用，不靠口头想象隐藏对象。 查看对应图解状态同一输入视图 a视图 bquery 编码器key 编码器qk⁺ = k₅L = −ln p⁺比较用旧字典k₁ … k₄q 与所有候选比较k⁺ 是本轮正确项梯度 → θqEMAk₅ 入队；k₁ 退队k₂ k₃ k₄ k₅下一批的字典按原文算法 1：更新 query，再平均 key 参数并维护下一批的字典。","evidenceIdentity":"用可指认的key身份解释历史复用，不靠口头想象隐藏对象。 查看对应图解状态","images":[{"src":"/assets/display/9a66faeae8dd2c354bc56482.webp","alt":""}]},"previousBlock":"b-f4b153b977e41f4781101f7c","nextBlock":"b-da0cfb6881ed0240e8b90afa","prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-da0cfb6881ed0240e8b90afa","type":"h2","scope":"moco-point-at-the-figure-dialogue","url":"https://codingai-lec03.pages.dev/course/blocks/moco-point-at-the-figure-dialogue.html#b-da0cfb6881ed0240e8b90afa","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"模拟问答：旧 key 还能比较吗？","number":1},"sectionTitle":"模拟问答：旧 key 还能比较吗？","currentQuestion":"模拟问答：旧 key 还能比较吗？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-014075331eed073b430d0909","nextBlock":null,"prerequisites":[{"id":"b-88d1ee9ab1d748201810a088","url":"#b-88d1ee9ab1d748201810a088","role":"section-title"},{"id":"b-13d6885b67e12d97a23788dd","url":"#b-13d6885b67e12d97a23788dd","role":"problem-context"}],"relatedBlocks":[{"id":"b-014075331eed073b430d0909","url":"#b-014075331eed073b430d0909","relation":"figure-and-explanation"}]}]}