{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"批内候选数为什么和 batch 绑在一起？","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.md","pages":[{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","unit":"怎样增加候选，同时让历史表示仍可比较？","unitId":"i-dictionary-and-updates","chapter":"I","number":1}],"aliases":[{"oldId":"b-ba4da3ecd438e4cbfd4c88f6","targetId":"b-ba4da3ecd438e4cbfd4c88f6","page":"reading-opening","operation":"move","beforeSHA256":"ceb58f01664839a052966e13279ca0ceecec30ecb0ff47933d3d88be9e89b15d","afterSHA256":"ceb58f01664839a052966e13279ca0ceecec30ecb0ff47933d3d88be9e89b15d","reason":"Same source body in authored semantic page"},{"oldId":"b-5f4b4855b8018ffd766c35c6","targetId":"b-5f4b4855b8018ffd766c35c6","page":"reading-opening","operation":"move","beforeSHA256":"628eb576124e20c5dc90d67d278cd3144ad68ffd5693c89d66f7ad111a740ed5","afterSHA256":"628eb576124e20c5dc90d67d278cd3144ad68ffd5693c89d66f7ad111a740ed5","reason":"Same source body in authored semantic page"},{"oldId":"b-b7d6b3b6b2c151cf698e630c","targetId":"b-b7d6b3b6b2c151cf698e630c","page":"reading-opening","operation":"move","beforeSHA256":"95073852f3df20e91190c6add6e91d49dde3e9b17de611bf7d850b13b530aeeb","afterSHA256":"95073852f3df20e91190c6add6e91d49dde3e9b17de611bf7d850b13b530aeeb","reason":"Same source body in authored semantic page"},{"oldId":"b-9ed79f4e79c41d50624a1a89","targetId":"b-9ed79f4e79c41d50624a1a89","page":"reading-opening","operation":"move","beforeSHA256":"26b5f0d4034962334575c7e782ca88475df8c258fcb4345ad6e145fe542f5add","afterSHA256":"26b5f0d4034962334575c7e782ca88475df8c258fcb4345ad6e145fe542f5add","reason":"Same source body in authored semantic page"},{"oldId":"b-9072d42a300831ff4935a875","targetId":"b-9072d42a300831ff4935a875","page":"reading-opening","operation":"move","beforeSHA256":"efe1853e876877caa9b416f10709ecc7d24da2143d33782e26efe19e7d4474c9","afterSHA256":"efe1853e876877caa9b416f10709ecc7d24da2143d33782e26efe19e7d4474c9","reason":"Same source body in authored semantic page"},{"oldId":"b-4e3119d9659ed5229dee768f","targetId":"b-4e3119d9659ed5229dee768f","page":"reading-opening","operation":"move","beforeSHA256":"22c57eee2950a19966a805f8a1e07e36ed9b336a981fe6314abdc76e8e6bc702","afterSHA256":"22c57eee2950a19966a805f8a1e07e36ed9b336a981fe6314abdc76e8e6bc702","reason":"Same source body in authored semantic page"},{"oldId":"b-60ed9cc703c6c0bc05a3a58c","targetId":"b-60ed9cc703c6c0bc05a3a58c","page":"reading-opening","operation":"rewrite","beforeSHA256":"b3edb6e9f1d075da6d444009bc8a3e125cf37b57f5ab06a55c43b9241ec5d1fb","afterSHA256":"b3f06fe6e586306f1f2295b463877e3489cb33fc7fd36ea6337203510787d48d","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-0c115393206c78463bcfb990","targetId":"b-0c115393206c78463bcfb990","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","type":"h1","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-ba4da3ecd438e4cbfd4c88f6","content":"# 批内候选数为什么和 batch 绑在一起？","sha256":"ceb58f01664839a052966e13279ca0ceecec30ecb0ff47933d3d88be9e89b15d","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-5f4b4855b8018ffd766c35c6","prerequisites":[{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]},{"id":"b-5f4b4855b8018ffd766c35c6","type":"p","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-5f4b4855b8018ffd766c35c6","content":"从同一个视图列表指认查询、自身和正例：4×4 到 6×6，每行候选从 3 项到 5 项。随后再把候选来源转为计算组织问题，为 MoCo 队列做铺垫。","sha256":"628eb576124e20c5dc90d67d278cd3144ad68ffd5693c89d66f7ad111a740ed5","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ba4da3ecd438e4cbfd4c88f6","nextBlock":"b-b7d6b3b6b2c151cf698e630c","prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]},{"id":"b-b7d6b3b6b2c151cf698e630c","type":"p","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-b7d6b3b6b2c151cf698e630c","content":"SimCLR 从 N 张原图产生 2N 个视图。P14 的两轴使用同一个视图列表，行是查询、列是候选。两张原图 A、B 对应 A1、A2、B1、B2；矩阵是 4×4。固定查询 A1，A1 自身排除，A2 是同源正例，B1、B2 是其余候选，所以每行比较 3 项。三张原图再加 C1、C2，矩阵成为 6×6，每行比较 5 项。一般地，每行排除自身后有 2N−1 项：1 个同源正例、2N−2 个其他候选。","sha256":"95073852f3df20e91190c6add6e91d49dde3e9b17de611bf7d850b13b530aeeb","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-5f4b4855b8018ffd766c35c6","nextBlock":"b-9ed79f4e79c41d50624a1a89","prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"},{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]},{"id":"b-9ed79f4e79c41d50624a1a89","type":"p","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-9ed79f4e79c41d50624a1a89","content":"主对角用灰色斜线排除，绿色格表示同一原图的另一个视图。若采用“两组不同视图分别作为两轴”的另一种排法，正例可以在对角；不能离开轴身份，仅凭绿色位置判断矩阵的含义。","sha256":"26b5f0d4034962334575c7e782ca88475df8c258fcb4345ad6e145fe542f5add","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b7d6b3b6b2c151cf698e630c","nextBlock":"b-9072d42a300831ff4935a875","prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"},{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]},{"id":"b-9072d42a300831ff4935a875","type":"p","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-9072d42a300831ff4935a875","content":"这些候选都由当前参数编码，彼此处于同一轮表示空间；代价是本步要处理更多图像与反向传播状态，完整的两两比较矩阵随视图数平方增加。增大 N 也会改变优化行为，并不只是免费加几个负例。","sha256":"efe1853e876877caa9b416f10709ecc7d24da2143d33782e26efe19e7d4474c9","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9ed79f4e79c41d50624a1a89","nextBlock":"b-4e3119d9659ed5229dee768f","prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"},{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]},{"id":"b-4e3119d9659ed5229dee768f","type":"p","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-4e3119d9659ed5229dee768f","content":"MoCo 的提问是：能否复用前几批已经算好的 key，让候选数超过当前 batch？这是下一方法的资源取舍，不是在 SimCLR 这一次比较中直接把旧键塞进矩阵。保存向量通常比保存整个图像计算图轻，但旧向量来自旧参数。下一页的队列解决“从哪里拿更多候选”，再下一页的 EMA 处理“拿来的旧候选是否仍可比较”。","sha256":"22c57eee2950a19966a805f8a1e07e36ed9b336a981fe6314abdc76e8e6bc702","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9072d42a300831ff4935a875","nextBlock":"b-60ed9cc703c6c0bc05a3a58c","prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"},{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]},{"id":"b-60ed9cc703c6c0bc05a3a58c","type":"figure","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-60ed9cc703c6c0bc05a3a58c","content":"图内标签与数值：P14 同一视图列表的 6×6 对比矩阵：自身排除、同源正例与其他候选；全部来自当前批次；能否复用历史特征？；3 张原图 → 6 个视图；原图 A；A1；A2；原图 B；B1；B2；原图 C；C1；C2；两轴使用同一个视图列表；行表示查询；A1；+；A2；+；B1；+；B2；+；C1；+；C2；+；A1；A2；B1；B2；C1；C2；固定查询 A1；A1：自身，排除；A2：同源正例；B1、B2、C1、C2：其余候选；每行：1 + 4 = 5 项；自身排除；同源正例；2N−2 个其余候选；当前编码 6 个视图；完整比较矩阵 6 × 6（随 2N 平方增长）。；下一方法：MoCo 的取舍；先问能否保存已算的 key；复用旧向量，减少本轮编码；候选数与当前批次能否解耦？；旧键还足够可比吗？\n\n两轴同列六个视图；A1 排除自身，A2 是同源正例，B1、B2、C1、C2 是四个其他候选。历史字典是下一方法的资源取舍。 查看对应图解状态\n\n- 全部来自当前批次\n- 能否复用历史特征？\n- 3 张原图 → 6 个视图\n- 原图 A\n- A1\n- A2\n- 原图 B\n- B1\n- B2\n- 原图 C\n- C1\n- C2\n- 两轴使用同一个视图列表；行表示查询\n- A1\n- +\n- A2\n- +\n- B1\n- +\n- B2\n- +\n- C1\n- +\n- C2\n- +\n- A1\n- A2\n- B1\n- B2\n- C1\n- C2\n- 固定查询 A1\n- A1：自身，排除\n- A2：同源正例\n- B1、B2、C1、C2：其余候选\n- 每行：1 + 4 = 5 项\n- 自身排除\n- 同源正例\n- 2N−2 个其余候选\n- 当前编码 6 个视图；完整比较矩阵 6 × 6（随 2N 平方增长）。\n- 下一方法：MoCo 的取舍\n- 先问能否保存已算的 key\n- 复用旧向量，减少本轮编码\n- 候选数与当前批次能否解耦？\n- 旧键还足够可比吗？两轴同列六个视图；A1 排除自身，A2 是同源正例，B1、B2、C1、C2 是四个其他候选。历史字典是下一方法的资源取舍。 [查看对应图解状态](/#/l3-r2-p14/1)[查看完整图解](/#/l3-r2-p14/1)","sha256":"b3f06fe6e586306f1f2295b463877e3489cb33fc7fd36ea6337203510787d48d","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"两轴同列六个视图；A1 排除自身，A2 是同源正例，B1、B2、C1、C2 是四个其他候选。历史字典是下一方法的资源取舍。 查看对应图解状态","labels":["全部来自当前批次","能否复用历史特征？","3 张原图 → 6 个视图","原图 A","A1","A2","原图 B","B1","B2","原图 C","C1","C2","两轴使用同一个视图列表；行表示查询","A1","+","A2","+","B1","+","B2","+","C1","+","C2","+","A1","A2","B1","B2","C1","C2","固定查询 A1","A1：自身，排除","A2：同源正例","B1、B2、C1、C2：其余候选","每行：1 + 4 = 5 项","自身排除","同源正例","2N−2 个其余候选","当前编码 6 个视图；完整比较矩阵 6 × 6（随 2N 平方增长）。","下一方法：MoCo 的取舍","先问能否保存已算的 key","复用旧向量，减少本轮编码","候选数与当前批次能否解耦？","旧键还足够可比吗？"],"description":"两轴同列六个视图；A1 排除自身，A2 是同源正例，B1、B2、C1、C2 是四个其他候选。历史字典是下一方法的资源取舍。 查看对应图解状态全部来自当前批次能否复用历史特征？3 张原图 → 6 个视图原图 AA1A2原图 BB1B2原图 CC1C2两轴使用同一个视图列表；行表示查询A1+A2+B1+B2+C1+C2+A1A2B1B2C1C2固定查询 A1A1：自身，排除A2：同源正例B1、B2、C1、C2：其余候选每行：1 + 4 = 5 项自身排除同源正例2N−2 个其余候选当前编码 6 个视图；完整比较矩阵 6 × 6（随 2N 平方增长）。下一方法：MoCo 的取舍先问能否保存已算的 key复用旧向量，减少本轮编码候选数与当前批次能否解耦？旧键还足够可比吗？","evidenceIdentity":"两轴同列六个视图；A1 排除自身，A2 是同源正例，B1、B2、C1、C2 是四个其他候选。历史字典是下一方法的资源取舍。 查看对应图解状态","images":[{"src":"/assets/display/9a66faeae8dd2c354bc56482.webp","alt":""},{"src":"/assets/display/8757616f3f92bbccd2d8db9c.webp","alt":""},{"src":"/assets/display/2e37eb55c23251072aa0d535.webp","alt":""}]},"previousBlock":"b-4e3119d9659ed5229dee768f","nextBlock":"b-0c115393206c78463bcfb990","prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"},{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0c115393206c78463bcfb990","type":"h2","scope":"current-batch-dictionary-cost","url":"https://codingai-lec03.pages.dev/course/blocks/current-batch-dictionary-cost.html#b-0c115393206c78463bcfb990","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-dictionary-and-updates","title":"怎样增加候选，同时让历史表示仍可比较？"},"page":{"id":"reading-opening","title":"批内候选数为什么和 batch 绑在一起？","number":1},"sectionTitle":"批内候选数为什么和 batch 绑在一起？","currentQuestion":"批内候选数为什么和 batch 绑在一起？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-60ed9cc703c6c0bc05a3a58c","nextBlock":null,"prerequisites":[{"id":"b-ba4da3ecd438e4cbfd4c88f6","url":"#b-ba4da3ecd438e4cbfd4c88f6","role":"section-title"},{"id":"b-5f4b4855b8018ffd766c35c6","url":"#b-5f4b4855b8018ffd766c35c6","role":"problem-context"}],"relatedBlocks":[{"id":"b-60ed9cc703c6c0bc05a3a58c","url":"#b-60ed9cc703c6c0bc05a3a58c","relation":"figure-and-explanation"}]}]}