{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"为什么队列需要缓慢变化的键编码器？","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.md","pages":[{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","unit":"为什么队列需要缓慢变化的键编码器？","unitId":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","chapter":"course","number":1}],"aliases":[{"oldId":"b-2eed584728351ce3d578aada","targetId":"b-2eed584728351ce3d578aada","page":"reading-opening","operation":"move","beforeSHA256":"88a04c9ecb442ccaa48afb715979a4f7ec256d11b69e583a238c98ab9d60e977","afterSHA256":"88a04c9ecb442ccaa48afb715979a4f7ec256d11b69e583a238c98ab9d60e977","reason":"Same source body in authored semantic page"},{"oldId":"b-136fccd6ce160b3cf602fece","targetId":"b-136fccd6ce160b3cf602fece","page":"reading-opening","operation":"move","beforeSHA256":"f60fcc0eedabb53764d71fa0a4af41ddf71a67bf00ef423c2a9319f48bed5c42","afterSHA256":"f60fcc0eedabb53764d71fa0a4af41ddf71a67bf00ef423c2a9319f48bed5c42","reason":"Same source body in authored semantic page"},{"oldId":"b-bf53a43c5bd0898a80ac3f7e","targetId":"b-bf53a43c5bd0898a80ac3f7e","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-648dead8269a9be99d47abe7","targetId":"b-648dead8269a9be99d47abe7","page":"reading-opening","operation":"move","beforeSHA256":"2a269d38480454fbc554b17f926927738416555e36045ec5fe468ba056454f5d","afterSHA256":"2a269d38480454fbc554b17f926927738416555e36045ec5fe468ba056454f5d","reason":"Same source body in authored semantic page"},{"oldId":"b-67afb62391c6ba995bd55440","targetId":"b-67afb62391c6ba995bd55440","page":"reading-opening","operation":"move","beforeSHA256":"c0f850f1b9199cde720c00a9fb1ab7cf8483b6e0541f5ab456b864fbba5da898","afterSHA256":"c0f850f1b9199cde720c00a9fb1ab7cf8483b6e0541f5ab456b864fbba5da898","reason":"Same source body in authored semantic page"},{"oldId":"b-470d36e4215f45c063d1172f","targetId":"b-470d36e4215f45c063d1172f","page":"reading-opening","operation":"move","beforeSHA256":"daf75b849add3482a2db67571a0277844e13e7354863bc30b3a0f2c624b605e0","afterSHA256":"daf75b849add3482a2db67571a0277844e13e7354863bc30b3a0f2c624b605e0","reason":"Same source body in authored semantic page"},{"oldId":"b-95e2585efb2ea7ff7e2dd7c9","targetId":"b-95e2585efb2ea7ff7e2dd7c9","page":"reading-opening","operation":"move","beforeSHA256":"735e86e63616ec89089d1b0695dd71e506a94498f2831a357002ca1d5329c8e8","afterSHA256":"735e86e63616ec89089d1b0695dd71e506a94498f2831a357002ca1d5329c8e8","reason":"Same source body in authored semantic page"},{"oldId":"b-4408ca6e298203c68ff73819","targetId":"b-4408ca6e298203c68ff73819","page":"reading-opening","operation":"move","beforeSHA256":"2449c8cac78957471c08dcab9b04ea03d40235c5eb9e43d9e76bcee0536b013a","afterSHA256":"2449c8cac78957471c08dcab9b04ea03d40235c5eb9e43d9e76bcee0536b013a","reason":"Same source body in authored semantic page"},{"oldId":"b-0140284720acffbbfe6358f0","targetId":"b-0140284720acffbbfe6358f0","page":"reading-opening","operation":"move","beforeSHA256":"567ba52bdaed9193421a349652115e6d4ba0c8ab9fa42ca8503aaafcb9978809","afterSHA256":"567ba52bdaed9193421a349652115e6d4ba0c8ab9fa42ca8503aaafcb9978809","reason":"Same source body in authored semantic page"},{"oldId":"b-25ff7bd84baafbac258e6425","targetId":"b-25ff7bd84baafbac258e6425","page":"reading-opening","operation":"move","beforeSHA256":"544e4bd80fa6b33bfb0b5569ce89f243d22424485d04f9258ef42efb7da800db","afterSHA256":"544e4bd80fa6b33bfb0b5569ce89f243d22424485d04f9258ef42efb7da800db","reason":"Same source body in authored semantic page"},{"oldId":"block-180ef4ed3ff34c0eaafec7bde6f86a8f--为什么队列需要缓慢变化的键编码器","targetId":"b-67afb62391c6ba995bd55440","operation":"retain-source-identity"}],"blocks":[{"id":"b-2eed584728351ce3d578aada","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-2eed584728351ce3d578aada","content":"[LAB 07 · MoCo：历史 key、队列与两种更新](/course/notebooks/0d1b8f6fbf6a476c93ffcc169f2ec1f4.html)","sha256":"88a04c9ecb442ccaa48afb715979a4f7ec256d11b69e583a238c98ab9d60e977","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-136fccd6ce160b3cf602fece","prerequisites":[],"relatedBlocks":[]},{"id":"b-136fccd6ce160b3cf602fece","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-136fccd6ce160b3cf602fece","content":"# 为什么队列需要缓慢变化的键编码器？","sha256":"f60fcc0eedabb53764d71fa0a4af41ddf71a67bf00ef423c2a9319f48bed5c42","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2eed584728351ce3d578aada","nextBlock":"b-bf53a43c5bd0898a80ac3f7e","prerequisites":[{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-bf53a43c5bd0898a80ac3f7e","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-bf53a43c5bd0898a80ac3f7e","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-136fccd6ce160b3cf602fece","nextBlock":"b-648dead8269a9be99d47abe7","prerequisites":[{"id":"b-136fccd6ce160b3cf602fece","url":"#b-136fccd6ce160b3cf602fece","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-648dead8269a9be99d47abe7","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-648dead8269a9be99d47abe7","content":"真实FIFO键、EMA、梯度更新及低于随机基线的保留结果","sha256":"2a269d38480454fbc554b17f926927738416555e36045ec5fe468ba056454f5d","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-bf53a43c5bd0898a80ac3f7e","nextBlock":"b-67afb62391c6ba995bd55440","prerequisites":[{"id":"b-136fccd6ce160b3cf602fece","url":"#b-136fccd6ce160b3cf602fece","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-67afb62391c6ba995bd55440","type":"h2","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-67afb62391c6ba995bd55440","content":"## 为什么队列需要缓慢变化的键编码器？","sha256":"c0f850f1b9199cde720c00a9fb1ab7cf8483b6e0541f5ab456b864fbba5da898","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-648dead8269a9be99d47abe7","nextBlock":"b-470d36e4215f45c063d1172f","prerequisites":[{"id":"b-136fccd6ce160b3cf602fece","url":"#b-136fccd6ce160b3cf602fece","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-470d36e4215f45c063d1172f","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-470d36e4215f45c063d1172f","content":"批内方法的键都由当前参数生成。FIFO 让字典容量与当前批大小分开，但旧键来自较早的参数；若表示空间每步剧烈旋转，今天的查询与昨天的键就难以比较。MoCo 用","sha256":"daf75b849add3482a2db67571a0277844e13e7354863bc30b3a0f2c624b605e0","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-67afb62391c6ba995bd55440","nextBlock":"b-95e2585efb2ea7ff7e2dd7c9","prerequisites":[{"id":"b-67afb62391c6ba995bd55440","url":"#b-67afb62391c6ba995bd55440","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-95e2585efb2ea7ff7e2dd7c9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-95e2585efb2ea7ff7e2dd7c9","content":"$\\theta_k\\leftarrow m\\theta_k+(1-m)\\theta_q$","sha256":"735e86e63616ec89089d1b0695dd71e506a94498f2831a357002ca1d5329c8e8","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"\\theta_k\\leftarrow m\\theta_k+(1-m)\\theta_q","definitionContext":"为什么队列需要缓慢变化的键编码器？"}],"previousBlock":"b-470d36e4215f45c063d1172f","nextBlock":"b-4408ca6e298203c68ff73819","prerequisites":[{"id":"b-67afb62391c6ba995bd55440","url":"#b-67afb62391c6ba995bd55440","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-4408ca6e298203c68ff73819","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-4408ca6e298203c68ff73819","content":"减缓键编码器变化。下面的教学实现 m=0.95，队列容量240、批大小60；query 和 key 初始权重相同。每步先对 key 做 EMA，再在 `no_grad` 中编码第二视图；查询梯度只更新 query；当次键以 detach 状态入队，最旧的键出队。第一次队列为空，采用当前批 detached keys 热身；后续正例放第0列，队列是负例列。","sha256":"2449c8cac78957471c08dcab9b04ea03d40235c5eb9e43d9e76bcee0536b013a","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-95e2585efb2ea7ff7e2dd7c9","nextBlock":"b-0140284720acffbbfe6358f0","prerequisites":[{"id":"b-67afb62391c6ba995bd55440","url":"#b-67afb62391c6ba995bd55440","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0140284720acffbbfe6358f0","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-0140284720acffbbfe6358f0","content":"为避免一个源图跨 epoch 再出现时被当作自己的负例，队列同时保存源图 ID，并屏蔽同源旧键。**同类别的不同源图仍可能是假负例**；这正是实例目标与语义类别不同的地方。此实现没有BatchNorm，因此没有多设备BN信息泄漏处理；论文工程细节不能据此宣称已全部实现。","sha256":"567ba52bdaed9193421a349652115e6d4ba0c8ab9fa42ca8503aaafcb9978809","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-4408ca6e298203c68ff73819","nextBlock":"b-25ff7bd84baafbac258e6425","prerequisites":[{"id":"b-67afb62391c6ba995bd55440","url":"#b-67afb62391c6ba995bd55440","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-25ff7bd84baafbac258e6425","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f.html#b-25ff7bd84baafbac258e6425","content":"主讲按原论文Algorithm 1展示先计算本批key、更新query、再EMA的流程；下面代码在本批key前做EMA。二者时序应分别按各自实现追踪，详见 [把一次学习走通，也读清实现顺序](/course/blocks/moco-one-step-paper-and-code.html)。这不是把旧队列重新编码。","sha256":"544e4bd80fa6b33bfb0b5569ce89f243d22424485d04f9258ef42efb7da800db","chapter":{"id":"course","title":"为什么队列需要缓慢变化的键编码器？"},"unit":{"id":"notebook-blocks/180ef4ed3ff34c0eaafec7bde6f86a8f","title":"为什么队列需要缓慢变化的键编码器？"},"page":{"id":"reading-opening","title":"为什么队列需要缓慢变化的键编码器？","number":1},"sectionTitle":"为什么队列需要缓慢变化的键编码器？","currentQuestion":"为什么队列需要缓慢变化的键编码器？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0140284720acffbbfe6358f0","nextBlock":null,"prerequisites":[{"id":"b-67afb62391c6ba995bd55440","url":"#b-67afb62391c6ba995bd55440","role":"section-title"},{"id":"b-2eed584728351ce3d578aada","url":"#b-2eed584728351ce3d578aada","role":"problem-context"}],"relatedBlocks":[]}]}