{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"图像条件怎样进入一个可变长度答案","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.md","pages":[{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":1}],"aliases":[{"oldId":"b-a45e14a26881cb8895fc42af","targetId":"b-a45e14a26881cb8895fc42af","page":"reading-opening","operation":"move","beforeSHA256":"d7e4f04d3b05e2e23ba37fcde7d80a552e4e714fe53ad7bb958021ae3c89e5ff","afterSHA256":"d7e4f04d3b05e2e23ba37fcde7d80a552e4e714fe53ad7bb958021ae3c89e5ff","reason":"Same source body in authored semantic page"},{"oldId":"b-c3c6bc5725280317600f4b43","targetId":"b-c3c6bc5725280317600f4b43","page":"reading-opening","operation":"move","beforeSHA256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","afterSHA256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","reason":"Same source body in authored semantic page"},{"oldId":"b-643e1a8529508d46c44f16ac","targetId":"b-643e1a8529508d46c44f16ac","page":"reading-opening","operation":"move","beforeSHA256":"73eb3160c6f026832b06ee63a280f8b54b7256670cc0346425cd8cb92efbd6e1","afterSHA256":"73eb3160c6f026832b06ee63a280f8b54b7256670cc0346425cd8cb92efbd6e1","reason":"Same source body in authored semantic page"},{"oldId":"b-50ffdbe36207738d4b5f7820","targetId":"b-50ffdbe36207738d4b5f7820","page":"reading-opening","operation":"move","beforeSHA256":"963b8cb4593a865d67c3b515ce7a59eda4fde2d526bf1fab3eedc10e8f3bbe72","afterSHA256":"963b8cb4593a865d67c3b515ce7a59eda4fde2d526bf1fab3eedc10e8f3bbe72","reason":"Same source body in authored semantic page"},{"oldId":"b-6f8bdf28a9c982f79a10649c","targetId":"b-6f8bdf28a9c982f79a10649c","page":"reading-opening","operation":"move","beforeSHA256":"38ac8dae0e03cd1374618f0f96fa050817f6b4ffb826dbe1ca1835086656d5be","afterSHA256":"38ac8dae0e03cd1374618f0f96fa050817f6b4ffb826dbe1ca1835086656d5be","reason":"Same source body in authored semantic page"},{"oldId":"b-e7c700fedca1f244d2de0cc3","targetId":"b-e7c700fedca1f244d2de0cc3","page":"reading-opening","operation":"move","beforeSHA256":"0c037e1d30329f16596e9cd66a94534e7997988977ea286a9927664b3809a313","afterSHA256":"0c037e1d30329f16596e9cd66a94534e7997988977ea286a9927664b3809a313","reason":"Same source body in authored semantic page"},{"oldId":"b-430c1930eab2cc6d5eb3c451","targetId":"b-430c1930eab2cc6d5eb3c451","page":"reading-opening","operation":"rewrite","beforeSHA256":"249b367d20b13e9b0fc6d450ed99bbe50784f4054e3a6a3f53d27d148cdb97f0","afterSHA256":"6ea7ce4296fa105bc4f8ac7468e98d6ca38f30724865e7912c23906bfb307295","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-72d2703549c59312b9bb629e","targetId":"b-72d2703549c59312b9bb629e","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-a45e14a26881cb8895fc42af","type":"h1","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-a45e14a26881cb8895fc42af","content":"# 图像条件怎样进入一个可变长度答案","sha256":"d7e4f04d3b05e2e23ba37fcde7d80a552e4e714fe53ad7bb958021ae3c89e5ff","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-c3c6bc5725280317600f4b43","prerequisites":[{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]},{"id":"b-c3c6bc5725280317600f4b43","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-c3c6bc5725280317600f4b43","content":"先沿画面指出视觉特征进入哪里，再指出已有文字与下一词目标。连续说明展开 shifted target、EOS、PAD 和实际生成，LAB13 提供已保存的实验结果。理解本页可先把文字预测器作为输入输出模块；整数移位只演示目标构造，完整图像描述还需要图像条件与实际生成。","sha256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a45e14a26881cb8895fc42af","nextBlock":"b-643e1a8529508d46c44f16ac","prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]},{"id":"b-643e1a8529508d46c44f16ac","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-643e1a8529508d46c44f16ac","content":"图像描述要建模的是 $p_\\theta(y_1,\\ldots,y_T\\mid I)$。链式法则将它分解为 $\\prod_t p_\\theta(y_t\\mid y_{<t},I)$：每一步都在图像和此前文字的条件下预测后续文字。这个分解是概率关系，不要求本讲先学完一种具体文字网络。","sha256":"73eb3160c6f026832b06ee63a280f8b54b7256670cc0346425cd8cb92efbd6e1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"p_\\theta(y_1,\\ldots,y_T\\mid I)","definitionContext":"图像条件怎样进入一个可变长度答案"},{"latex":"\\prod_t p_\\theta(y_t\\mid y_{<t},I)","definitionContext":"图像条件怎样进入一个可变长度答案"}],"previousBlock":"b-c3c6bc5725280317600f4b43","nextBlock":"b-50ffdbe36207738d4b5f7820","prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"},{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]},{"id":"b-50ffdbe36207738d4b5f7820","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-50ffdbe36207738d4b5f7820","content":"指着 P51 的图看三个位置：图像编码器把像素变成视觉特征；文字预测器同时接收视觉条件与已有文字；目标位置从“红衣人”后的“拿着”，推进到“红衣人拿着”后的“球”。如果删除视觉路径，模型仍可能补出语法合理的句子，却失去了判断画面到底是谁拿球的信息来源。","sha256":"963b8cb4593a865d67c3b515ce7a59eda4fde2d526bf1fab3eedc10e8f3bbe72","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-643e1a8529508d46c44f16ac","nextBlock":"b-6f8bdf28a9c982f79a10649c","prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"},{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]},{"id":"b-6f8bdf28a9c982f79a10649c","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-6f8bdf28a9c982f79a10649c","content":"训练图文配对提供目标句子，而不是图片自行生成唯一文字标签。一幅图可以被说成“两个人站在一起”，也可以被说成“红衣人拿着球”。前者省略衣色，后者聚焦动作主体，两句都可能真实。训练学习的是这种条件下的表达分布；教师还需要说明任务要求概括、细节描述还是完整枚举。","sha256":"38ac8dae0e03cd1374618f0f96fa050817f6b4ffb826dbe1ca1835086656d5be","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-50ffdbe36207738d4b5f7820","nextBlock":"b-e7c700fedca1f244d2de0cc3","prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"},{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]},{"id":"b-e7c700fedca1f244d2de0cc3","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-e7c700fedca1f244d2de0cc3","content":"推理时没有参考句子可偷看。模型从开始符号出发，输出一个词，再把这个词接到自己的上下文，直到结束符号或长度上限。完整的实验必须有实际图像、可学习参数、词目标、优化和这条自由生成过程。网页上推进的参考词是教学目标示意；实际生成与视觉对照放在 LAB13。","sha256":"0c037e1d30329f16596e9cd66a94534e7997988977ea286a9927664b3809a313","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6f8bdf28a9c982f79a10649c","nextBlock":"b-430c1930eab2cc6d5eb3c451","prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"},{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]},{"id":"b-430c1930eab2cc6d5eb3c451","type":"figure","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-430c1930eab2cc6d5eb3c451","content":"图内标签与数值：视觉编码器；图像 → 特征；两条合理描述；“两个人站在一起。”；“红衣人拿着篮球。”；文字预测器；允许看到的前文 + 视觉；红衣人 拿着；球；下一个目标；图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。\n\n图像条件、已有前缀与下一词目标各有独立位置；图上词语不是本轮模型实测。 查看对应图解状态\n\n- 视觉编码器\n- 图像 → 特征\n- 两条合理描述\n- “两个人站在一起。”\n- “红衣人拿着篮球。”\n- 文字预测器\n- 允许看到的前文 + 视觉\n- 红衣人 拿着\n- 球\n- 下一个目标\n- 图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。图像条件、已有前缀与下一词目标各有独立位置；图上词语不是本轮模型实测。 [查看对应图解状态](/#/l3-r2-p51/1)[查看完整图解](/#/l3-r2-p51/1)","sha256":"6ea7ce4296fa105bc4f8ac7468e98d6ca38f30724865e7912c23906bfb307295","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"图像条件、已有前缀与下一词目标各有独立位置；图上词语不是本轮模型实测。 查看对应图解状态","labels":["视觉编码器","图像 → 特征","两条合理描述","“两个人站在一起。”","“红衣人拿着篮球。”","文字预测器","允许看到的前文 + 视觉","红衣人 拿着","球","下一个目标","图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。"],"description":"图像条件、已有前缀与下一词目标各有独立位置；图上词语不是本轮模型实测。 查看对应图解状态视觉编码器图像 → 特征两条合理描述“两个人站在一起。”“红衣人拿着篮球。”文字预测器允许看到的前文 + 视觉红衣人 拿着球下一个目标图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。","evidenceIdentity":"图像条件、已有前缀与下一词目标各有独立位置；图上词语不是本轮模型实测。 查看对应图解状态","images":[{"src":"/assets/display/512610a019c560ff0384e86a.webp","alt":""}]},"previousBlock":"b-e7c700fedca1f244d2de0cc3","nextBlock":"b-72d2703549c59312b9bb629e","prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"},{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-72d2703549c59312b9bb629e","type":"h2","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/blocks/caption-conditioned-token.html#b-72d2703549c59312b9bb629e","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"图像条件怎样进入一个可变长度答案","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"图像条件怎样进入一个可变长度答案","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-430c1930eab2cc6d5eb3c451","nextBlock":null,"prerequisites":[{"id":"b-a45e14a26881cb8895fc42af","url":"#b-a45e14a26881cb8895fc42af","role":"section-title"},{"id":"b-c3c6bc5725280317600f4b43","url":"#b-c3c6bc5725280317600f4b43","role":"problem-context"}],"relatedBlocks":[{"id":"b-430c1930eab2cc6d5eb3c451","url":"#b-430c1930eab2cc6d5eb3c451","relation":"figure-and-explanation"}]}]}