{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"从概率到字典更新","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.md","pages":[{"id":"reading-opening","title":"从概率到字典更新","unit":"从概率到字典更新","unitId":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","chapter":"course","number":1}],"aliases":[{"oldId":"b-bb98fce24e2c63e376639396","targetId":"b-bb98fce24e2c63e376639396","page":"reading-opening","operation":"move","beforeSHA256":"88a04c9ecb442ccaa48afb715979a4f7ec256d11b69e583a238c98ab9d60e977","afterSHA256":"88a04c9ecb442ccaa48afb715979a4f7ec256d11b69e583a238c98ab9d60e977","reason":"Same source body in authored semantic page"},{"oldId":"b-fa8e007f031a34d086dd1b7d","targetId":"b-fa8e007f031a34d086dd1b7d","page":"reading-opening","operation":"move","beforeSHA256":"aa351107807d5fdd175cef061d8dfe0bc0de3dc7d325e7177748e44a0ee00b73","afterSHA256":"aa351107807d5fdd175cef061d8dfe0bc0de3dc7d325e7177748e44a0ee00b73","reason":"Same source body in authored semantic page"},{"oldId":"b-d3071f35d2522b268d87b977","targetId":"b-d3071f35d2522b268d87b977","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-b62fbc3d03035baa3ec77e38","targetId":"b-b62fbc3d03035baa3ec77e38","page":"reading-opening","operation":"move","beforeSHA256":"2a269d38480454fbc554b17f926927738416555e36045ec5fe468ba056454f5d","afterSHA256":"2a269d38480454fbc554b17f926927738416555e36045ec5fe468ba056454f5d","reason":"Same source body in authored semantic page"},{"oldId":"b-a81db73bbdafef885dd3499c","targetId":"b-a81db73bbdafef885dd3499c","page":"reading-opening","operation":"move","beforeSHA256":"ad3252a44d79e4087efe23d5bea992c60eedf5a6038f4c815284f90ef6dfb01a","afterSHA256":"ad3252a44d79e4087efe23d5bea992c60eedf5a6038f4c815284f90ef6dfb01a","reason":"Same source body in authored semantic page"},{"oldId":"b-9802f00cc37a7b63286ba562","targetId":"b-9802f00cc37a7b63286ba562","page":"reading-opening","operation":"move","beforeSHA256":"04381f6370956c58d0864cf84728d04b4c98814c2101236e49217b6099204eb9","afterSHA256":"04381f6370956c58d0864cf84728d04b4c98814c2101236e49217b6099204eb9","reason":"Same source body in authored semantic page"},{"oldId":"b-5958f2a41452147ca554cf57","targetId":"b-5958f2a41452147ca554cf57","page":"reading-opening","operation":"move","beforeSHA256":"1f7e2a886cdafdc45b2221dbb40ee2276777f66502510e8a5256d0918f722da4","afterSHA256":"1f7e2a886cdafdc45b2221dbb40ee2276777f66502510e8a5256d0918f722da4","reason":"Same source body in authored semantic page"},{"oldId":"block-feb682a46fff4f59a3edf8c6c2b6634a--从概率到字典更新","targetId":"b-a81db73bbdafef885dd3499c","operation":"retain-source-identity"}],"blocks":[{"id":"b-bb98fce24e2c63e376639396","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-bb98fce24e2c63e376639396","content":"[LAB 07 · MoCo：历史 key、队列与两种更新](/course/notebooks/0d1b8f6fbf6a476c93ffcc169f2ec1f4.html)","sha256":"88a04c9ecb442ccaa48afb715979a4f7ec256d11b69e583a238c98ab9d60e977","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-fa8e007f031a34d086dd1b7d","prerequisites":[],"relatedBlocks":[]},{"id":"b-fa8e007f031a34d086dd1b7d","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-fa8e007f031a34d086dd1b7d","content":"# 从概率到字典更新","sha256":"aa351107807d5fdd175cef061d8dfe0bc0de3dc7d325e7177748e44a0ee00b73","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-bb98fce24e2c63e376639396","nextBlock":"b-d3071f35d2522b268d87b977","prerequisites":[{"id":"b-bb98fce24e2c63e376639396","url":"#b-bb98fce24e2c63e376639396","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d3071f35d2522b268d87b977","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-d3071f35d2522b268d87b977","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-fa8e007f031a34d086dd1b7d","nextBlock":"b-b62fbc3d03035baa3ec77e38","prerequisites":[{"id":"b-fa8e007f031a34d086dd1b7d","url":"#b-fa8e007f031a34d086dd1b7d","role":"section-title"},{"id":"b-bb98fce24e2c63e376639396","url":"#b-bb98fce24e2c63e376639396","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b62fbc3d03035baa3ec77e38","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-b62fbc3d03035baa3ec77e38","content":"真实FIFO键、EMA、梯度更新及低于随机基线的保留结果","sha256":"2a269d38480454fbc554b17f926927738416555e36045ec5fe468ba056454f5d","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d3071f35d2522b268d87b977","nextBlock":"b-a81db73bbdafef885dd3499c","prerequisites":[{"id":"b-fa8e007f031a34d086dd1b7d","url":"#b-fa8e007f031a34d086dd1b7d","role":"section-title"},{"id":"b-bb98fce24e2c63e376639396","url":"#b-bb98fce24e2c63e376639396","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a81db73bbdafef885dd3499c","type":"h2","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-a81db73bbdafef885dd3499c","content":"## 从概率到字典更新","sha256":"ad3252a44d79e4087efe23d5bea992c60eedf5a6038f4c815284f90ef6dfb01a","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b62fbc3d03035baa3ec77e38","nextBlock":"b-9802f00cc37a7b63286ba562","prerequisites":[{"id":"b-fa8e007f031a34d086dd1b7d","url":"#b-fa8e007f031a34d086dd1b7d","role":"section-title"},{"id":"b-bb98fce24e2c63e376639396","url":"#b-bb98fce24e2c63e376639396","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9802f00cc37a7b63286ba562","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-9802f00cc37a7b63286ba562","content":"查询 q=(1,0)，归一化后的第一个键最接近它。温度0.1时，其 softmax 概率约0.999939；温度0.5时约0.859767。温度改变的是已有相似度差距在概率中的放大，不会改变此处的键排序。","sha256":"04381f6370956c58d0864cf84728d04b4c98814c2101236e49217b6099204eb9","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a81db73bbdafef885dd3499c","nextBlock":"b-5958f2a41452147ca554cf57","prerequisites":[{"id":"b-a81db73bbdafef885dd3499c","url":"#b-a81db73bbdafef885dd3499c","role":"section-title"},{"id":"b-bb98fce24e2c63e376639396","url":"#b-bb98fce24e2c63e376639396","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-5958f2a41452147ca554cf57","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a.html#b-5958f2a41452147ca554cf57","content":"FIFO 容量4，从 a,b,c,d 入队 e,f 后得到 c,d,e,f。键参数 θk=(0,2)、查询参数 θq=(2,4)，m=0.9 的一次 EMA 是 (0.2,2.2)。EMA 不是把两个模型的输出向量平均，也不是对键编码器额外反向传播。","sha256":"1f7e2a886cdafdc45b2221dbb40ee2276777f66502510e8a5256d0918f722da4","chapter":{"id":"course","title":"从概率到字典更新"},"unit":{"id":"notebook-blocks/feb682a46fff4f59a3edf8c6c2b6634a","title":"从概率到字典更新"},"page":{"id":"reading-opening","title":"从概率到字典更新","number":1},"sectionTitle":"从概率到字典更新","currentQuestion":"从概率到字典更新","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9802f00cc37a7b63286ba562","nextBlock":null,"prerequisites":[{"id":"b-a81db73bbdafef885dd3499c","url":"#b-a81db73bbdafef885dd3499c","role":"section-title"},{"id":"b-bb98fce24e2c63e376639396","url":"#b-bb98fce24e2c63e376639396","role":"problem-context"}],"relatedBlocks":[]}]}