{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.md","pages":[{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","unit":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","unitId":"notebook-blocks/9af985696da54136a73aa78d650b93c5","chapter":"course","number":1}],"aliases":[{"oldId":"b-9ec4ff6a640e75c6cc148257","targetId":"b-9ec4ff6a640e75c6cc148257","page":"reading-opening","operation":"move","beforeSHA256":"343fb71807f077069b41c6495a8824ea4d353c740175b5775ccc5145b3bf19ca","afterSHA256":"343fb71807f077069b41c6495a8824ea4d353c740175b5775ccc5145b3bf19ca","reason":"Same source body in authored semantic page"},{"oldId":"b-01b08a9e4f52bd461f1b736b","targetId":"b-01b08a9e4f52bd461f1b736b","page":"reading-opening","operation":"move","beforeSHA256":"1fb3a599fe674666c1a908547c02b69aaf327e412003ca496aded4f9215a8dd0","afterSHA256":"1fb3a599fe674666c1a908547c02b69aaf327e412003ca496aded4f9215a8dd0","reason":"Same source body in authored semantic page"},{"oldId":"b-badd2c07204b1f8d83092ec9","targetId":"b-badd2c07204b1f8d83092ec9","page":"reading-opening","operation":"move","beforeSHA256":"42cba31deccba2a350b9b23bd0ca1dcaa4c4c5cb9d35c773e1fcb89124788ccd","afterSHA256":"42cba31deccba2a350b9b23bd0ca1dcaa4c4c5cb9d35c773e1fcb89124788ccd","reason":"Same source body in authored semantic page"},{"oldId":"b-b15e5ea0e35fc0c528516cd5","targetId":"b-b15e5ea0e35fc0c528516cd5","page":"reading-opening","operation":"move","beforeSHA256":"41c70c2c07b82678ff0a60f2d009b6afa750c5c691d109e754a69abf23868b50","afterSHA256":"41c70c2c07b82678ff0a60f2d009b6afa750c5c691d109e754a69abf23868b50","reason":"Same source body in authored semantic page"},{"oldId":"b-7be653182d981e012a990326","targetId":"b-7be653182d981e012a990326","page":"reading-opening","operation":"move","beforeSHA256":"7aca821f1c7bffbb560cbea620602884dbd6c22947e99b30387a9ae006a78714","afterSHA256":"7aca821f1c7bffbb560cbea620602884dbd6c22947e99b30387a9ae006a78714","reason":"Same source body in authored semantic page"},{"oldId":"b-42ed4dce7e13bc8675aeab67","targetId":"b-42ed4dce7e13bc8675aeab67","page":"reading-opening","operation":"move","beforeSHA256":"2fe5de42360d0416004495225148d4789a58190d6d5d6f8ccf8d6f676cc7db8f","afterSHA256":"2fe5de42360d0416004495225148d4789a58190d6d5d6f8ccf8d6f676cc7db8f","reason":"Same source body in authored semantic page"},{"oldId":"b-4823b6572ecaf4c1e99024e5","targetId":"b-4823b6572ecaf4c1e99024e5","page":"reading-opening","operation":"move","beforeSHA256":"a4199dc2105b7fcf2d31b7486c23469b3c2adfd5e951d87c8810ebe4fe3e0734","afterSHA256":"a4199dc2105b7fcf2d31b7486c23469b3c2adfd5e951d87c8810ebe4fe3e0734","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-9ec4ff6a640e75c6cc148257","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-9ec4ff6a640e75c6cc148257","content":"[CLS 03 · 视觉与语言：任务、训练与接口](/course/notebooks/680da245145447dd88983aabd136fd24.html)","sha256":"343fb71807f077069b41c6495a8824ea4d353c740175b5775ccc5145b3bf19ca","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-01b08a9e4f52bd461f1b736b","prerequisites":[],"relatedBlocks":[]},{"id":"b-01b08a9e4f52bd461f1b736b","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-01b08a9e4f52bd461f1b736b","content":"# 图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sha256":"1fb3a599fe674666c1a908547c02b69aaf327e412003ca496aded4f9215a8dd0","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9ec4ff6a640e75c6cc148257","nextBlock":"b-badd2c07204b1f8d83092ec9","prerequisites":[{"id":"b-9ec4ff6a640e75c6cc148257","url":"#b-9ec4ff6a640e75c6cc148257","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-badd2c07204b1f8d83092ec9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-badd2c07204b1f8d83092ec9","content":"主线阅读与优先实验","sha256":"42cba31deccba2a350b9b23bd0ca1dcaa4c4c5cb9d35c773e1fcb89124788ccd","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-01b08a9e4f52bd461f1b736b","nextBlock":"b-b15e5ea0e35fc0c528516cd5","prerequisites":[{"id":"b-01b08a9e4f52bd461f1b736b","url":"#b-01b08a9e4f52bd461f1b736b","role":"section-title"},{"id":"b-9ec4ff6a640e75c6cc148257","url":"#b-9ec4ff6a640e75c6cc148257","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b15e5ea0e35fc0c528516cd5","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-b15e5ea0e35fc0c528516cd5","content":"caption历史、VQA、CLIP候选、关系控制、GLIP/LLaVA与生成桥梁","sha256":"41c70c2c07b82678ff0a60f2d009b6afa750c5c691d109e754a69abf23868b50","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-badd2c07204b1f8d83092ec9","nextBlock":"b-7be653182d981e012a990326","prerequisites":[{"id":"b-01b08a9e4f52bd461f1b736b","url":"#b-01b08a9e4f52bd461f1b736b","role":"section-title"},{"id":"b-9ec4ff6a640e75c6cc148257","url":"#b-9ec4ff6a640e75c6cc148257","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7be653182d981e012a990326","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-7be653182d981e012a990326","content":"图像描述学习 $p(y_t\\mid I,y_{<t})$：视觉特征与已有文字一起决定下一词。训练时配对描述给出目标；推理时从开始符号出发，只能接续自己的输出，直到结束符号。今天先把这条接口看清，循环网络内部留第五讲。","sha256":"7aca821f1c7bffbb560cbea620602884dbd6c22947e99b30387a9ae006a78714","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"p(y_t\\mid I,y_{<t})","definitionContext":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"}],"previousBlock":"b-b15e5ea0e35fc0c528516cd5","nextBlock":"b-42ed4dce7e13bc8675aeab67","prerequisites":[{"id":"b-01b08a9e4f52bd461f1b736b","url":"#b-01b08a9e4f52bd461f1b736b","role":"section-title"},{"id":"b-9ec4ff6a640e75c6cc148257","url":"#b-9ec4ff6a640e75c6cc148257","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-42ed4dce7e13bc8675aeab67","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-42ed4dce7e13bc8675aeab67","content":"[返回图像条件的课堂图](/course/pages/l3-r2-p51.html)。完整的 shifted target、EOS/PAD 和保留集生成对照归并到 [CLS 09 · 图像条件输出：描述与问答](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/c9f328a00ef44ecc90804e47276a7811?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=c9f328a00ef44ecc90804e47276a7811&utm_term=get_notebook) 与 [LAB 13 · 看图生成一句话：训练、解码与消融](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/3da47f3ee3554ee0a36e0bb6a2fdc61f?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=3da47f3ee3554ee0a36e0bb6a2fdc61f&utm_term=get_notebook)；本册不复制两份训练说明。","sha256":"2fe5de42360d0416004495225148d4789a58190d6d5d6f8ccf8d6f676cc7db8f","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7be653182d981e012a990326","nextBlock":"b-4823b6572ecaf4c1e99024e5","prerequisites":[{"id":"b-01b08a9e4f52bd461f1b736b","url":"#b-01b08a9e4f52bd461f1b736b","role":"section-title"},{"id":"b-9ec4ff6a640e75c6cc148257","url":"#b-9ec4ff6a640e75c6cc148257","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-4823b6572ecaf4c1e99024e5","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/9af985696da54136a73aa78d650b93c5.html#b-4823b6572ecaf4c1e99024e5","content":"一张图可有多种真实描述。“桌上有杯子”与“左侧红杯靠近书”聚焦信息不同。先说明任务要求概括还是详细描述，再判断遗漏是否构成错误。[图像条件怎样进入一个可变长度答案](/course/blocks/caption-conditioned-token.html) · [已保存的条件生成实验：视觉输入确实改变了句子](/course/blocks/caption-visual-controls.html)。","sha256":"a4199dc2105b7fcf2d31b7486c23469b3c2adfd5e951d87c8810ebe4fe3e0734","chapter":{"id":"course","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"unit":{"id":"notebook-blocks/9af985696da54136a73aa78d650b93c5","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词"},"page":{"id":"reading-opening","title":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","number":1},"sectionTitle":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","currentQuestion":"图像描述学习 $p(yt\\mid I,y{<t})$：视觉特征与已有文字一起决定下一词","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-42ed4dce7e13bc8675aeab67","nextBlock":null,"prerequisites":[{"id":"b-01b08a9e4f52bd461f1b736b","url":"#b-01b08a9e4f52bd461f1b736b","role":"section-title"},{"id":"b-9ec4ff6a640e75c6cc148257","url":"#b-9ec4ff6a640e75c6cc148257","role":"problem-context"}],"relatedBlocks":[]}]}