{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.md","pages":[{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":1}],"aliases":[{"oldId":"b-a6b8ff669b8347b75c07ad55","targetId":"b-a6b8ff669b8347b75c07ad55","page":"reading-opening","operation":"move","beforeSHA256":"c3f0d2502f7d6510a5de99a32b32212b1ff58b8db86db2e7e0dee31c484649fb","afterSHA256":"c3f0d2502f7d6510a5de99a32b32212b1ff58b8db86db2e7e0dee31c484649fb","reason":"Same source body in authored semantic page"},{"oldId":"b-7231092fb6644c720dc1e6e2","targetId":"b-7231092fb6644c720dc1e6e2","page":"reading-opening","operation":"move","beforeSHA256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","afterSHA256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","reason":"Same source body in authored semantic page"},{"oldId":"b-770dd510da3cdd6bcd93584d","targetId":"b-770dd510da3cdd6bcd93584d","page":"reading-opening","operation":"move","beforeSHA256":"dd0cbe5a9f571aff92f5177d64ed0ecef2a16c1713f740eea8cb5312f7a9f650","afterSHA256":"dd0cbe5a9f571aff92f5177d64ed0ecef2a16c1713f740eea8cb5312f7a9f650","reason":"Same source body in authored semantic page"},{"oldId":"b-b280fcc5a2cb42b8e9189520","targetId":"b-b280fcc5a2cb42b8e9189520","page":"reading-opening","operation":"rewrite","beforeSHA256":"791f8a35e18efd7bbc9a959937b6e461f00d5dc05cf985f8a69f8fa52f66f71c","afterSHA256":"7cf82523725f8dc1dbfc45a269f6e423480bce0858c9bf86dcb2bc5f68cfb9ab","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-3bbc813ab4fa082ecd361479","targetId":"b-3bbc813ab4fa082ecd361479","page":"reading-opening","operation":"move","beforeSHA256":"43952b0c901e562451395edc1a8bdeef87074554a5ad6cc25045b1d31c451f16","afterSHA256":"43952b0c901e562451395edc1a8bdeef87074554a5ad6cc25045b1d31c451f16","reason":"Same source body in authored semantic page"},{"oldId":"b-978c31ecb320635388802139","targetId":"b-978c31ecb320635388802139","page":"reading-opening","operation":"move","beforeSHA256":"abec1e77f4ee30f312e664a3c45a315e4bfe246e526b1dd7bb578311d2ada770","afterSHA256":"abec1e77f4ee30f312e664a3c45a315e4bfe246e526b1dd7bb578311d2ada770","reason":"Same source body in authored semantic page"},{"oldId":"b-91b93763d65a4175e52d0f04","targetId":"b-91b93763d65a4175e52d0f04","page":"reading-opening","operation":"move","beforeSHA256":"37cf03728db1dab4711e27ff47809b2be238f681a3a44e01bcb88c5744536edc","afterSHA256":"37cf03728db1dab4711e27ff47809b2be238f681a3a44e01bcb88c5744536edc","reason":"Same source body in authored semantic page"},{"oldId":"b-8b8fd69b3da11461cafd9a83","targetId":"b-8b8fd69b3da11461cafd9a83","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-a6b8ff669b8347b75c07ad55","type":"h1","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-a6b8ff669b8347b75c07ad55","content":"# 训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sha256":"c3f0d2502f7d6510a5de99a32b32212b1ff58b8db86db2e7e0dee31c484649fb","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-7231092fb6644c720dc1e6e2","prerequisites":[{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7231092fb6644c720dc1e6e2","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-7231092fb6644c720dc1e6e2","content":"先沿画面指出视觉特征进入哪里，再指出已有文字与下一词目标。连续说明展开 shifted target、EOS、PAD 和实际生成，LAB13 提供已保存的实验结果。理解本页可先把文字预测器作为输入输出模块；整数移位只演示目标构造，完整图像描述还需要图像条件与实际生成。","sha256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a6b8ff669b8347b75c07ad55","nextBlock":"b-770dd510da3cdd6bcd93584d","prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"}],"relatedBlocks":[]},{"id":"b-770dd510da3cdd6bcd93584d","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-770dd510da3cdd6bcd93584d","content":"假设一条参考句为 `a red square`，词序列写成 `[BOS,a,red,square,EOS]`。训练输入去掉最后一项，目标去掉第一项。两者一一对齐：","sha256":"dd0cbe5a9f571aff92f5177d64ed0ecef2a16c1713f740eea8cb5312f7a9f650","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7231092fb6644c720dc1e6e2","nextBlock":"b-b280fcc5a2cb42b8e9189520","prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"},{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b280fcc5a2cb42b8e9189520","type":"table","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-b280fcc5a2cb42b8e9189520","content":"| 已允许看到的文字 | 当前目标 |\n| --- | --- |\n| BOS | a |\n| BOS a | red |\n| BOS a red | square |\n| BOS a red square | EOS |","sha256":"7cf82523725f8dc1dbfc45a269f6e423480bce0858c9bf86dcb2bc5f68cfb9ab","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["已允许看到的文字","当前目标"],"rows":[["已允许看到的文字","当前目标"],["BOS","a"],["BOS a","red"],["BOS a red","square"],["BOS a red square","EOS"]]},"previousBlock":"b-770dd510da3cdd6bcd93584d","nextBlock":"b-3bbc813ab4fa082ecd361479","prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"},{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3bbc813ab4fa082ecd361479","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-3bbc813ab4fa082ecd361479","content":"如果批中还有更长句子，就给短句末尾补 PAD，使张量长度一致。PAD 只是存储约定，不是要生成的文字；EOS 却是有效的停止目标。掩掉 PAD 时把 EOS 一起掩掉，会抹去何时结束的监督。","sha256":"43952b0c901e562451395edc1a8bdeef87074554a5ad6cc25045b1d31c451f16","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b280fcc5a2cb42b8e9189520","nextBlock":"b-978c31ecb320635388802139","prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"},{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-978c31ecb320635388802139","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-978c31ecb320635388802139","content":"词损失可写为 $L=-\\sum_{i,t}m_{it}\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})/\\sum_{i,t}m_{it}$，其中 $m_{it}=1$ 表示目标不是 PAD。分母按有效词计数，不按补齐后的格数计数。LAB13 的 `prefix=Y[:,:-1]`、`targets=Y[:,1:]` 和 `ignore_index=PAD` 对应这三个选择。","sha256":"abec1e77f4ee30f312e664a3c45a315e4bfe246e526b1dd7bb578311d2ada770","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"L=-\\sum_{i,t}m_{it}\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})/\\sum_{i,t}m_{it}","definitionContext":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置"},{"latex":"m_{it}=1","definitionContext":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置"}],"previousBlock":"b-3bbc813ab4fa082ecd361479","nextBlock":"b-91b93763d65a4175e52d0f04","prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"},{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-91b93763d65a4175e52d0f04","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-91b93763d65a4175e52d0f04","content":"训练时喂真实前缀叫 teacher forcing。推理时前缀来自模型自己：若先误说 blue，后续就必须在 blue 的条件下继续，不能偷偷换回 red。因此保留集的逐词交叉熵与自由生成的整句正确率要分别读；两者评价的输入条件不同。","sha256":"37cf03728db1dab4711e27ff47809b2be238f681a3a44e01bcb88c5744536edc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-978c31ecb320635388802139","nextBlock":"b-8b8fd69b3da11461cafd9a83","prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"},{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8b8fd69b3da11461cafd9a83","type":"h2","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/blocks/caption-shift-and-stop.html#b-8b8fd69b3da11461cafd9a83","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","number":1},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-91b93763d65a4175e52d0f04","nextBlock":null,"prerequisites":[{"id":"b-a6b8ff669b8347b75c07ad55","url":"#b-a6b8ff669b8347b75c07ad55","role":"section-title"},{"id":"b-7231092fb6644c720dc1e6e2","url":"#b-7231092fb6644c720dc1e6e2","role":"problem-context"}],"relatedBlocks":[]}]}