{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"从整图相关到局部对应：名词与关系不是同一层信息","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.md","pages":[{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":1}],"aliases":[{"oldId":"b-de718f1ef00055a4f970a3a9","targetId":"b-de718f1ef00055a4f970a3a9","page":"reading-opening","operation":"move","beforeSHA256":"e615c2d7aa71695d1aab7c8ebd21ca7115911313db7e5939a29c12f09f462232","afterSHA256":"e615c2d7aa71695d1aab7c8ebd21ca7115911313db7e5939a29c12f09f462232","reason":"Same source body in authored semantic page"},{"oldId":"b-881825382c3b3763d6ad5779","targetId":"b-881825382c3b3763d6ad5779","page":"reading-opening","operation":"move","beforeSHA256":"dd1eb9eed20bff23b21561095fe860aa6cd309cba0f9dcc4dd6d1a61491e77d6","afterSHA256":"dd1eb9eed20bff23b21561095fe860aa6cd309cba0f9dcc4dd6d1a61491e77d6","reason":"Same source body in authored semantic page"},{"oldId":"b-a69244e39f20e07cf414e81c","targetId":"b-a69244e39f20e07cf414e81c","page":"reading-opening","operation":"move","beforeSHA256":"fcb2d4df06ee0911a7c0b6fa3a64259c4e275e40ed10912934f233ab1bd44e4d","afterSHA256":"fcb2d4df06ee0911a7c0b6fa3a64259c4e275e40ed10912934f233ab1bd44e4d","reason":"Same source body in authored semantic page"},{"oldId":"b-c62645fedce9160b9cbffad8","targetId":"b-c62645fedce9160b9cbffad8","page":"reading-opening","operation":"move","beforeSHA256":"753167e916d99733d10317a382bb23d2e9d7c82ad02c291fbefb3baefc81b303","afterSHA256":"753167e916d99733d10317a382bb23d2e9d7c82ad02c291fbefb3baefc81b303","reason":"Same source body in authored semantic page"},{"oldId":"b-5ef0d40557720119eddd1a9e","targetId":"b-5ef0d40557720119eddd1a9e","page":"reading-opening","operation":"move","beforeSHA256":"a789d8b2a0148c8d95268149cbeb62a417b9f42ebef95300851b7c7da1ea9651","afterSHA256":"a789d8b2a0148c8d95268149cbeb62a417b9f42ebef95300851b7c7da1ea9651","reason":"Same source body in authored semantic page"},{"oldId":"b-4c782ebe0ea1e4c6b2aa6b15","targetId":"b-4c782ebe0ea1e4c6b2aa6b15","page":"reading-opening","operation":"move","beforeSHA256":"0d25f6fba136681c0086f981761ef0e25643e37172a7e5968ed5d26704874802","afterSHA256":"0d25f6fba136681c0086f981761ef0e25643e37172a7e5968ed5d26704874802","reason":"Same source body in authored semantic page"},{"oldId":"b-6cb7a471bcfa6ec4b44752df","targetId":"b-6cb7a471bcfa6ec4b44752df","page":"reading-opening","operation":"rewrite","beforeSHA256":"71fc6856e83736dd0235c4ae895e5a364e6fb9e2fbc5c35d1091ac43678d9fc3","afterSHA256":"ef0fdf6dd373dd8e20fca069ddda58891c948ed3563a9b02529262fc80e1fc47","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-558229f32cbab2c2d830675f","targetId":"b-558229f32cbab2c2d830675f","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-de718f1ef00055a4f970a3a9","type":"h1","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-de718f1ef00055a4f970a3a9","content":"# 从整图相关到局部对应：名词与关系不是同一层信息","sha256":"e615c2d7aa71695d1aab7c8ebd21ca7115911313db7e5939a29c12f09f462232","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-881825382c3b3763d6ad5779","prerequisites":[{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]},{"id":"b-881825382c3b3763d6ad5779","type":"p","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-881825382c3b3763d6ad5779","content":"在同一图上分开读实体和关系。三个名词可以各自对应框，“拿着”却必须绑定正确人物与球。局部区域使语言有可以指向的对象，但关系不由找到两个对象自动获得。图上的人工标注说明任务，已保存的像素规则实验则给出可检查的输出链。","sha256":"dd1eb9eed20bff23b21561095fe860aa6cd309cba0f9dcc4dd6d1a61491e77d6","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-de718f1ef00055a4f970a3a9","nextBlock":"b-a69244e39f20e07cf414e81c","prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]},{"id":"b-a69244e39f20e07cf414e81c","type":"p","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-a69244e39f20e07cf414e81c","content":"整图相似度可以说“这幅画面与两人持球有关”，却没有说明“红衣人”对应哪块像素、“球”是哪一个区域。把第二部分学过的区域表示取出来，句中的片段就有了可以对齐的视觉对象。","sha256":"fcb2d4df06ee0911a7c0b6fa3a64259c4e275e40ed10912934f233ab1bd44e4d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-881825382c3b3763d6ad5779","nextBlock":"b-c62645fedce9160b9cbffad8","prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"},{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]},{"id":"b-c62645fedce9160b9cbffad8","type":"p","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-c62645fedce9160b9cbffad8","content":"P53 中，红衣人、蓝衣人和球分别有候选框；“红衣人”和“球”可以各自指向区域。但“拿着”不是第四种独立物体，它描述两个实体之间的关系。即使候选框都正确，如果把关系连到另一位人物，整句依然错误。实体识别和关系绑定因而要分别检查。","sha256":"753167e916d99733d10317a382bb23d2e9d7c82ad02c291fbefb3baefc81b303","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a69244e39f20e07cf414e81c","nextBlock":"b-5ef0d40557720119eddd1a9e","prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"},{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]},{"id":"b-5ef0d40557720119eddd1a9e","type":"p","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-5ef0d40557720119eddd1a9e","content":"Karpathy 与 Li Fei-Fei 的区域—语言对齐工作从图像与句子资料学习局部跨模态对应，并进一步生成区域描述；论文在图文检索及区域描述上给出实证。其价值是让局部对应成为可学习、可复用的对象，而不是把整图压成一个类别之后才补语言。课堂图上的框和关系线是人工标注，不冒充该论文或本课程模型的预测。","sha256":"a789d8b2a0148c8d95268149cbeb62a417b9f42ebef95300851b7c7da1ea9651","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c62645fedce9160b9cbffad8","nextBlock":"b-4c782ebe0ea1e4c6b2aa6b15","prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"},{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]},{"id":"b-4c782ebe0ea1e4c6b2aa6b15","type":"p","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-4c782ebe0ea1e4c6b2aa6b15","content":"做一个最小自查：把句子改为“蓝衣人在红衣人旁边”，两个实体框可能保持不变，关系应随语句变化；改为“拿着球的人”，输出目标变成受关系约束的人物。区域表示提供候选，语言条件决定怎样选择，二者缺一不可。","sha256":"0d25f6fba136681c0086f981761ef0e25643e37172a7e5968ed5d26704874802","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-5ef0d40557720119eddd1a9e","nextBlock":"b-6cb7a471bcfa6ec4b44752df","prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"},{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]},{"id":"b-6cb7a471bcfa6ec4b44752df","type":"figure","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-6cb7a471bcfa6ec4b44752df","content":"图内标签与数值：红衣人；拿着；球；实体：对应局部内容；关系：连接人物与球；区域表示 + 语言片段 → 学习更细的对应；教学框与关系弧线；区域对齐方法并不因此自动获得全部动作关系。\n\n实体线连接名词与框，关系弧线连接人物和球；二者承担不同信息。 查看对应图解状态\n\n- 红衣人\n- 拿着\n- 球\n- 实体：对应局部内容\n- 关系：连接人物与球\n- 区域表示 + 语言片段 → 学习更细的对应\n- 教学框与关系弧线；区域对齐方法并不因此自动获得全部动作关系。实体线连接名词与框，关系弧线连接人物和球；二者承担不同信息。 [查看对应图解状态](/#/l3-r2-p53/0)[查看完整图解](/#/l3-r2-p53/0)","sha256":"ef0fdf6dd373dd8e20fca069ddda58891c948ed3563a9b02529262fc80e1fc47","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"实体线连接名词与框，关系弧线连接人物和球；二者承担不同信息。 查看对应图解状态","labels":["红衣人","拿着","球","实体：对应局部内容","关系：连接人物与球","区域表示 + 语言片段 → 学习更细的对应","教学框与关系弧线；区域对齐方法并不因此自动获得全部动作关系。"],"description":"实体线连接名词与框，关系弧线连接人物和球；二者承担不同信息。 查看对应图解状态红衣人拿着球实体：对应局部内容关系：连接人物与球区域表示 + 语言片段 → 学习更细的对应教学框与关系弧线；区域对齐方法并不因此自动获得全部动作关系。","evidenceIdentity":"实体线连接名词与框，关系弧线连接人物和球；二者承担不同信息。 查看对应图解状态","images":[{"src":"/assets/display/512610a019c560ff0384e86a.webp","alt":""}]},"previousBlock":"b-4c782ebe0ea1e4c6b2aa6b15","nextBlock":"b-558229f32cbab2c2d830675f","prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"},{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-558229f32cbab2c2d830675f","type":"h2","scope":"region-word-alignment","url":"https://codingai-lec03.pages.dev/course/blocks/region-word-alignment.html#b-558229f32cbab2c2d830675f","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"从整图相关到局部对应：名词与关系不是同一层信息","number":1},"sectionTitle":"从整图相关到局部对应：名词与关系不是同一层信息","currentQuestion":"从整图相关到局部对应：名词与关系不是同一层信息","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6cb7a471bcfa6ec4b44752df","nextBlock":null,"prerequisites":[{"id":"b-de718f1ef00055a4f970a3a9","url":"#b-de718f1ef00055a4f970a3a9","role":"section-title"},{"id":"b-881825382c3b3763d6ad5779","url":"#b-881825382c3b3763d6ad5779","role":"problem-context"}],"relatedBlocks":[{"id":"b-6cb7a471bcfa6ec4b44752df","url":"#b-6cb7a471bcfa6ec4b44752df","relation":"figure-and-explanation"}]}]}