{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"同一幅图像的四种任务契约","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.md","pages":[{"id":"reading-opening","title":"同一幅图像的四种任务契约","unit":"语言加入后，任务究竟改变了什么？","unitId":"language-task-interfaces","chapter":"III","number":1}],"aliases":[{"oldId":"b-b74503b3f54bcc86016f124f","targetId":"b-b74503b3f54bcc86016f124f","page":"reading-opening","operation":"move","beforeSHA256":"6307ee2691f3968e880a4d6ee8e1e8bc64b2d51f21b46996d538b0e291e7870c","afterSHA256":"6307ee2691f3968e880a4d6ee8e1e8bc64b2d51f21b46996d538b0e291e7870c","reason":"Same source body in authored semantic page"},{"oldId":"b-ab3277071db249ea560f20b5","targetId":"b-ab3277071db249ea560f20b5","page":"reading-opening","operation":"move","beforeSHA256":"e3630f7949680c0e12f08b38af40e9ff11a8e7133f218ea0bbc2b066b003241b","afterSHA256":"e3630f7949680c0e12f08b38af40e9ff11a8e7133f218ea0bbc2b066b003241b","reason":"Same source body in authored semantic page"},{"oldId":"b-f3163d3502393b87e6a79d32","targetId":"b-f3163d3502393b87e6a79d32","page":"reading-opening","operation":"move","beforeSHA256":"6d3f0ccf29f8486f89c006dcd39d2ae46a4818747e3dc66620dbe62b4d2bcef8","afterSHA256":"6d3f0ccf29f8486f89c006dcd39d2ae46a4818747e3dc66620dbe62b4d2bcef8","reason":"Same source body in authored semantic page"},{"oldId":"b-a13faffeed31eab956c54371","targetId":"b-a13faffeed31eab956c54371","page":"reading-opening","operation":"rewrite","beforeSHA256":"d4e6ff29020bc51e5156e88c0f0105eb8cce94525c3ce1a0069979cb08f3e8f1","afterSHA256":"6a58ead378f63fc6c7302948e82a938dbf19bbc4da2712206680ba660e973ae3","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-4c24ad6e6a8f5e40d3e8e607","targetId":"b-4c24ad6e6a8f5e40d3e8e607","page":"reading-opening","operation":"move","beforeSHA256":"ce971c80fc5973f61f0afd00e90986c4e787b467eb367d446448599d11b01c1c","afterSHA256":"ce971c80fc5973f61f0afd00e90986c4e787b467eb367d446448599d11b01c1c","reason":"Same source body in authored semantic page"},{"oldId":"b-12c0d32e9f5447decb0ac1af","targetId":"b-12c0d32e9f5447decb0ac1af","page":"reading-opening","operation":"move","beforeSHA256":"722df0f66abf5b4be284b5eaef034b6cd09a690a8d6141103a1d864f79510316","afterSHA256":"722df0f66abf5b4be284b5eaef034b6cd09a690a8d6141103a1d864f79510316","reason":"Same source body in authored semantic page"},{"oldId":"b-617dfbfe3b2ba0f2467345ec","targetId":"b-617dfbfe3b2ba0f2467345ec","page":"reading-opening","operation":"rewrite","beforeSHA256":"44c9209d0df8870782975ea3f4080cb1401167a664845f4ebe562066fdad6928","afterSHA256":"ec75ee6ad1fc6f1140e66672dd5e009c8efdfa187e6a57a5b392457ed92aaad1","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-a33276e46071389c887e2e91","targetId":"b-a33276e46071389c887e2e91","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-b74503b3f54bcc86016f124f","type":"h1","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-b74503b3f54bcc86016f124f","content":"# 同一幅图像的四种任务契约","sha256":"6307ee2691f3968e880a4d6ee8e1e8bc64b2d51f21b46996d538b0e291e7870c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-ab3277071db249ea560f20b5","prerequisites":[{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]},{"id":"b-ab3277071db249ea560f20b5","type":"p","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-ab3277071db249ea560f20b5","content":"第三部分仍从同一场景出发，只改变语言的作用。请先对照四个输出：句子、候选分数、区域、答案。它们都可以使用视觉和语言，却不是同一个任务换四个名字。读完本页，应能为任一请求说出输入、输出与目标来源，再按自己的问题进入后面的说明。","sha256":"e3630f7949680c0e12f08b38af40e9ff11a8e7133f218ea0bbc2b066b003241b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b74503b3f54bcc86016f124f","nextBlock":"b-f3163d3502393b87e6a79d32","prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]},{"id":"b-f3163d3502393b87e6a79d32","type":"p","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-f3163d3502393b87e6a79d32","content":"把两个人和一个球的同一幅图保持不动，可以提出四个不同任务。差别首先在输出，而不在模型名字。","sha256":"6d3f0ccf29f8486f89c006dcd39d2ae46a4818747e3dc66620dbe62b4d2bcef8","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ab3277071db249ea560f20b5","nextBlock":"b-a13faffeed31eab956c54371","prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"},{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]},{"id":"b-a13faffeed31eab956c54371","type":"table","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-a13faffeed31eab956c54371","content":"| 请求 | 实际输入 | 要交付的输出 | 训练材料应提供什么 |\n| --- | --- | --- | --- |\n| 描述画面 | 图像 | 一段新句子 | 图像与参考描述 |\n| 哪句话匹配 | 图像与文字候选 | 每个候选的分数或排名 | 已配对图文及对比关系 |\n| 找到拿球的人 | 图像与指代短语 | 对象身份及框/区域 | 短语与区域对应、定位目标 |\n| 谁拿着球 | 图像与问题 | 问题条件下的答案 | 图像、问题、答案 |","sha256":"6a58ead378f63fc6c7302948e82a938dbf19bbc4da2712206680ba660e973ae3","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["请求","实际输入","要交付的输出","训练材料应提供什么"],"rows":[["请求","实际输入","要交付的输出","训练材料应提供什么"],["描述画面","图像","一段新句子","图像与参考描述"],["哪句话匹配","图像与文字候选","每个候选的分数或排名","已配对图文及对比关系"],["找到拿球的人","图像与指代短语","对象身份及框/区域","短语与区域对应、定位目标"],["谁拿着球","图像与问题","问题条件下的答案","图像、问题、答案"]]},"previousBlock":"b-f3163d3502393b87e6a79d32","nextBlock":"b-4c24ad6e6a8f5e40d3e8e607","prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"},{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]},{"id":"b-4c24ad6e6a8f5e40d3e8e607","type":"p","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-4c24ad6e6a8f5e40d3e8e607","content":"“红衣人拿着球”可以同时是参考描述、匹配候选和问答的解释，但它们在训练中扮演的角色不同。描述模型要知道怎样产生这句话；匹配模型可以只在已经给出的两句话之间选择；grounding 还要输出坐标。一个很好的相似度分数不能直接替代这些不同输出。","sha256":"ce971c80fc5973f61f0afd00e90986c4e787b467eb367d446448599d11b01c1c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a13faffeed31eab956c54371","nextBlock":"b-12c0d32e9f5447decb0ac1af","prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"},{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]},{"id":"b-12c0d32e9f5447decb0ac1af","type":"p","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-12c0d32e9f5447decb0ac1af","content":"第二部分已经建立区域和像素接口。第三部分把语言加入之后，继续追问：语言在本次计算里是目标、监督来源还是任务条件？这三个角色可以共存。视觉助手收到一个问题、参考图像并生成回答时，问题是条件，回答是生成目标，而训练时的图文和指令数据是学习信号。","sha256":"722df0f66abf5b4be284b5eaef034b6cd09a690a8d6141103a1d864f79510316","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-4c24ad6e6a8f5e40d3e8e607","nextBlock":"b-617dfbfe3b2ba0f2467345ec","prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"},{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]},{"id":"b-617dfbfe3b2ba0f2467345ec","type":"figure","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-617dfbfe3b2ba0f2467345ec","content":"图内标签与数值：描述；红衣人拿着一个篮球。；匹配；A 红衣人拿球  /  B 蓝衣人拿球；指代；“拿球的人” → 人物区域；问答；谁拿着球？ → 红衣人；同一图像；句子 · 选择 · 区域 · 答案；场景与输出均为任务示意；本页没有运行描述、定位或问答模型。\n\n四种输出共用同一场景；这里的示范输出是人工定义的任务目标。 查看对应图解状态\n\n- 描述\n- 红衣人拿着一个篮球。\n- 匹配\n- A 红衣人拿球 / B 蓝衣人拿球\n- 指代\n- “拿球的人” → 人物区域\n- 问答\n- 谁拿着球？ → 红衣人\n- 同一图像\n- 句子 · 选择 · 区域 · 答案\n- 场景与输出均为任务示意；本页没有运行描述、定位或问答模型。四种输出共用同一场景；这里的示范输出是人工定义的任务目标。 [查看对应图解状态](/#/l3-r2-p49/0)[查看完整图解](/#/l3-r2-p49/0)","sha256":"ec75ee6ad1fc6f1140e66672dd5e009c8efdfa187e6a57a5b392457ed92aaad1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"四种输出共用同一场景；这里的示范输出是人工定义的任务目标。 查看对应图解状态","labels":["描述","红衣人拿着一个篮球。","匹配","A 红衣人拿球  /  B 蓝衣人拿球","指代","“拿球的人” → 人物区域","问答","谁拿着球？ → 红衣人","同一图像","句子 · 选择 · 区域 · 答案","场景与输出均为任务示意；本页没有运行描述、定位或问答模型。"],"description":"四种输出共用同一场景；这里的示范输出是人工定义的任务目标。 查看对应图解状态描述红衣人拿着一个篮球。匹配A 红衣人拿球 / B 蓝衣人拿球指代“拿球的人” → 人物区域问答谁拿着球？ → 红衣人同一图像句子 · 选择 · 区域 · 答案场景与输出均为任务示意；本页没有运行描述、定位或问答模型。","evidenceIdentity":"四种输出共用同一场景；这里的示范输出是人工定义的任务目标。 查看对应图解状态","images":[{"src":"/assets/display/512610a019c560ff0384e86a.webp","alt":""}]},"previousBlock":"b-12c0d32e9f5447decb0ac1af","nextBlock":"b-a33276e46071389c887e2e91","prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"},{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a33276e46071389c887e2e91","type":"h2","scope":"vl-task-contracts","url":"https://codingai-lec03.pages.dev/course/blocks/vl-task-contracts.html#b-a33276e46071389c887e2e91","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-task-interfaces","title":"语言加入后，任务究竟改变了什么？"},"page":{"id":"reading-opening","title":"同一幅图像的四种任务契约","number":1},"sectionTitle":"同一幅图像的四种任务契约","currentQuestion":"同一幅图像的四种任务契约","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-617dfbfe3b2ba0f2467345ec","nextBlock":null,"prerequisites":[{"id":"b-b74503b3f54bcc86016f124f","url":"#b-b74503b3f54bcc86016f124f","role":"section-title"},{"id":"b-ab3277071db249ea560f20b5","url":"#b-ab3277071db249ea560f20b5","role":"problem-context"}],"relatedBlocks":[{"id":"b-617dfbfe3b2ba0f2467345ec","url":"#b-617dfbfe3b2ba0f2467345ec","relation":"figure-and-explanation"}]}]}