{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"把对比损失算成一道选择题","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.md","pages":[{"id":"reading-opening","title":"把对比损失算成一道选择题","unit":"视图关系怎样成为学习目标，又会牺牲什么？","unitId":"i-view-objective","chapter":"I","number":1}],"aliases":[{"oldId":"b-cf8517271ec05a24a3d3d9bf","targetId":"b-cf8517271ec05a24a3d3d9bf","page":"reading-opening","operation":"move","beforeSHA256":"db966c933aec7d619fcafe33f105fe803c7ab3cb55d371724b473296f7e96e23","afterSHA256":"db966c933aec7d619fcafe33f105fe803c7ab3cb55d371724b473296f7e96e23","reason":"Same source body in authored semantic page"},{"oldId":"b-0787b4a2f77bc16508cbe050","targetId":"b-0787b4a2f77bc16508cbe050","page":"reading-opening","operation":"move","beforeSHA256":"617994cb9e9eba93f0bf3265ec9ed81f11bc03c1613f1ad98c4222386b3673c8","afterSHA256":"617994cb9e9eba93f0bf3265ec9ed81f11bc03c1613f1ad98c4222386b3673c8","reason":"Same source body in authored semantic page"},{"oldId":"b-0291dff741946cdd4465527f","targetId":"b-0291dff741946cdd4465527f","page":"reading-opening","operation":"move","beforeSHA256":"9c85b2a94e12be025a25906ccd3086fd9e23b04cd418d9dae7558fa656dda707","afterSHA256":"9c85b2a94e12be025a25906ccd3086fd9e23b04cd418d9dae7558fa656dda707","reason":"Same source body in authored semantic page"},{"oldId":"b-e2bdf0e9050d1e154ddeb0f8","targetId":"b-e2bdf0e9050d1e154ddeb0f8","page":"reading-opening","operation":"move","beforeSHA256":"ccf18d20e76ccb7263acbdd725b5435a8780de7aa0dbced13935f9a1805751d6","afterSHA256":"ccf18d20e76ccb7263acbdd725b5435a8780de7aa0dbced13935f9a1805751d6","reason":"Same source body in authored semantic page"},{"oldId":"b-e2c118789b2ec6d80b7684a5","targetId":"b-e2c118789b2ec6d80b7684a5","page":"reading-opening","operation":"move","beforeSHA256":"39863b658779ff6d27c44f9e8dab11bb04fcb26103fa8aea399c9833d66f0fb8","afterSHA256":"39863b658779ff6d27c44f9e8dab11bb04fcb26103fa8aea399c9833d66f0fb8","reason":"Same source body in authored semantic page"},{"oldId":"b-93960a48fbc3eae4aeb6bdf2","targetId":"b-93960a48fbc3eae4aeb6bdf2","page":"reading-opening","operation":"rewrite","beforeSHA256":"3c89243aad5bfa18f144bb44f437397b8d443184a4ad858d049d06e99dccea50","afterSHA256":"09f0d5e3db1336cc2aee12838c68b3d8d626df0b9193b9df070d27ba7b48e6ef","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-cf60fd7d1810d5bb8b1e5bf6","targetId":"b-cf60fd7d1810d5bb8b1e5bf6","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-cf8517271ec05a24a3d3d9bf","type":"h1","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-cf8517271ec05a24a3d3d9bf","content":"# 把对比损失算成一道选择题","sha256":"db966c933aec7d619fcafe33f105fe803c7ab3cb55d371724b473296f7e96e23","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-0787b4a2f77bc16508cbe050","prerequisites":[{"id":"b-0787b4a2f77bc16508cbe050","url":"#b-0787b4a2f77bc16508cbe050","role":"problem-context"}],"relatedBlocks":[{"id":"b-93960a48fbc3eae4aeb6bdf2","url":"#b-93960a48fbc3eae4aeb6bdf2","relation":"figure-and-explanation"}]},{"id":"b-0787b4a2f77bc16508cbe050","type":"p","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-0787b4a2f77bc16508cbe050","content":"这一页把相似关系变成可优化的概率与损失。分数、温度、softmax和负对数必须能对上同一组数。","sha256":"617994cb9e9eba93f0bf3265ec9ed81f11bc03c1613f1ad98c4222386b3673c8","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-cf8517271ec05a24a3d3d9bf","nextBlock":"b-0291dff741946cdd4465527f","prerequisites":[{"id":"b-cf8517271ec05a24a3d3d9bf","url":"#b-cf8517271ec05a24a3d3d9bf","role":"section-title"}],"relatedBlocks":[{"id":"b-93960a48fbc3eae4aeb6bdf2","url":"#b-93960a48fbc3eae4aeb6bdf2","relation":"figure-and-explanation"}]},{"id":"b-0291dff741946cdd4465527f","type":"p","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-0291dff741946cdd4465527f","content":"设查询的三个候选logit为2、1、0，第一项是已知正例。这些是**温度缩放后的分数**，不是三个原始余弦值。softmax分母为 $e^2+e^1+e^0\\approx11.107$，概率约为 $(0.6652,0.2447,0.0900)$，正例损失 $-\\ln0.6652\\approx0.4076$。","sha256":"9c85b2a94e12be025a25906ccd3086fd9e23b04cd418d9dae7558fa656dda707","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"e^2+e^1+e^0\\approx11.107","definitionContext":"把对比损失算成一道选择题"},{"latex":"(0.6652,0.2447,0.0900)","definitionContext":"把对比损失算成一道选择题"},{"latex":"-\\ln0.6652\\approx0.4076","definitionContext":"把对比损失算成一道选择题"}],"previousBlock":"b-0787b4a2f77bc16508cbe050","nextBlock":"b-e2bdf0e9050d1e154ddeb0f8","prerequisites":[{"id":"b-cf8517271ec05a24a3d3d9bf","url":"#b-cf8517271ec05a24a3d3d9bf","role":"section-title"},{"id":"b-0787b4a2f77bc16508cbe050","url":"#b-0787b4a2f77bc16508cbe050","role":"problem-context"}],"relatedBlocks":[{"id":"b-93960a48fbc3eae4aeb6bdf2","url":"#b-93960a48fbc3eae4aeb6bdf2","relation":"figure-and-explanation"}]},{"id":"b-e2bdf0e9050d1e154ddeb0f8","type":"p","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-e2bdf0e9050d1e154ddeb0f8","content":"一般式为 $L=-\\log\\{\\exp(s_+/\\tau)/\\sum_j\\exp(s_j/\\tau)\\}$，分母包含正例。提高正例相对分数会降低损失；把所有分数同时加一个常数不会改变概率，所以目标关注相对选择而非绝对分数大。","sha256":"ccf18d20e76ccb7263acbdd725b5435a8780de7aa0dbced13935f9a1805751d6","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"L=-\\log\\{\\exp(s_+/\\tau)/\\sum_j\\exp(s_j/\\tau)\\}","definitionContext":"把对比损失算成一道选择题"}],"previousBlock":"b-0291dff741946cdd4465527f","nextBlock":"b-e2c118789b2ec6d80b7684a5","prerequisites":[{"id":"b-cf8517271ec05a24a3d3d9bf","url":"#b-cf8517271ec05a24a3d3d9bf","role":"section-title"},{"id":"b-0787b4a2f77bc16508cbe050","url":"#b-0787b4a2f77bc16508cbe050","role":"problem-context"}],"relatedBlocks":[{"id":"b-93960a48fbc3eae4aeb6bdf2","url":"#b-93960a48fbc3eae4aeb6bdf2","relation":"figure-and-explanation"}]},{"id":"b-e2c118789b2ec6d80b7684a5","type":"p","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-e2c118789b2ec6d80b7684a5","content":"实际训练中分数由图像经编码器算出，损失反传到参数。手算这一行只解释目标，不是模型实验。接到LAB 07时，应沿真实图像→两视图→向量→logit→loss→参数变化逐项找到对应对象。","sha256":"39863b658779ff6d27c44f9e8dab11bb04fcb26103fa8aea399c9833d66f0fb8","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e2bdf0e9050d1e154ddeb0f8","nextBlock":"b-93960a48fbc3eae4aeb6bdf2","prerequisites":[{"id":"b-cf8517271ec05a24a3d3d9bf","url":"#b-cf8517271ec05a24a3d3d9bf","role":"section-title"},{"id":"b-0787b4a2f77bc16508cbe050","url":"#b-0787b4a2f77bc16508cbe050","role":"problem-context"}],"relatedBlocks":[{"id":"b-93960a48fbc3eae4aeb6bdf2","url":"#b-93960a48fbc3eae4aeb6bdf2","relation":"figure-and-explanation"}]},{"id":"b-93960a48fbc3eae4aeb6bdf2","type":"figure","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-93960a48fbc3eae4aeb6bdf2","content":"图内标签与数值：P10 学习表示机制图，状态 2；三候选教学算例 · 指定第 1 项为正例；相似度 s；除以温度 τ = 0.5；归一化概率 p；k⁺；1；2；0.665；k₁；0.5；1；0.245；k₂；0；0；0.090；pⱼ = exp(sⱼ / τ) / Σₗ exp(sₗ / τ)；L = −ln p⁺ = 0.408\n\n三个logit经过softmax，正例概率对应负对数损失。 查看对应图解状态\n\n- 三候选教学算例 · 指定第 1 项为正例\n- 相似度 s\n- 除以温度 τ = 0.5\n- 归一化概率 p\n- k⁺\n- 1\n- 2\n- 0.665\n- k₁\n- 0.5\n- 1\n- 0.245\n- k₂\n- 0\n- 0\n- 0.090\n- pⱼ = exp(sⱼ / τ) / Σₗ exp(sₗ / τ)\n- L = −ln p⁺ = 0.408三个logit经过softmax，正例概率对应负对数损失。 [查看对应图解状态](/#/l3-r2-p10/1)[查看完整图解](/#/l3-r2-p10/1)","sha256":"09f0d5e3db1336cc2aee12838c68b3d8d626df0b9193b9df070d27ba7b48e6ef","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"三个logit经过softmax，正例概率对应负对数损失。 查看对应图解状态","labels":["三候选教学算例 · 指定第 1 项为正例","相似度 s","除以温度 τ = 0.5","归一化概率 p","k⁺","1","2","0.665","k₁","0.5","1","0.245","k₂","0","0","0.090","pⱼ = exp(sⱼ / τ) / Σₗ exp(sₗ / τ)","L = −ln p⁺ = 0.408"],"description":"三个logit经过softmax，正例概率对应负对数损失。 查看对应图解状态三候选教学算例 · 指定第 1 项为正例相似度 s除以温度 τ = 0.5归一化概率 pk⁺120.665k₁0.510.245k₂000.090pⱼ = exp(sⱼ / τ) / Σₗ exp(sₗ / τ)L = −ln p⁺ = 0.408","evidenceIdentity":"三个logit经过softmax，正例概率对应负对数损失。 查看对应图解状态","images":[]},"previousBlock":"b-e2c118789b2ec6d80b7684a5","nextBlock":"b-cf60fd7d1810d5bb8b1e5bf6","prerequisites":[{"id":"b-cf8517271ec05a24a3d3d9bf","url":"#b-cf8517271ec05a24a3d3d9bf","role":"section-title"},{"id":"b-0787b4a2f77bc16508cbe050","url":"#b-0787b4a2f77bc16508cbe050","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-cf60fd7d1810d5bb8b1e5bf6","type":"h2","scope":"infonce-as-candidate-choice","url":"https://codingai-lec03.pages.dev/course/blocks/infonce-as-candidate-choice.html#b-cf60fd7d1810d5bb8b1e5bf6","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"把对比损失算成一道选择题","number":1},"sectionTitle":"把对比损失算成一道选择题","currentQuestion":"把对比损失算成一道选择题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-93960a48fbc3eae4aeb6bdf2","nextBlock":null,"prerequisites":[{"id":"b-cf8517271ec05a24a3d3d9bf","url":"#b-cf8517271ec05a24a3d3d9bf","role":"section-title"},{"id":"b-0787b4a2f77bc16508cbe050","url":"#b-0787b4a2f77bc16508cbe050","role":"problem-context"}],"relatedBlocks":[{"id":"b-93960a48fbc3eae4aeb6bdf2","url":"#b-93960a48fbc3eae4aeb6bdf2","relation":"figure-and-explanation"}]}]}