{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"VQA：问题决定此刻需要哪一项信息","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html","markdown":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.md","pages":[{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":1},{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":2}],"aliases":[{"oldId":"explanation","targetId":"vqa-image-question-controls","operation":"split-explanation-range"},{"oldId":"b-274ef0b79375737f880dd447","targetId":"b-274ef0b79375737f880dd447","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","afterSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","reason":"Same source body in authored semantic page"},{"oldId":"b-25a847d665e3181abddd827d","targetId":"b-25a847d665e3181abddd827d","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"19cc0322b455f4f4f375c0f9d8ca856db923c8bfb82a36655eec2226a3b67669","afterSHA256":"19cc0322b455f4f4f375c0f9d8ca856db923c8bfb82a36655eec2226a3b67669","reason":"Same source body in authored semantic page"},{"oldId":"b-9c22db528963c756bbf806ae","targetId":"b-9c22db528963c756bbf806ae","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"e694c4320e7af6f266b80de8b9d0453b3a6861ef55dfd0b426416c93d755292d","afterSHA256":"e694c4320e7af6f266b80de8b9d0453b3a6861ef55dfd0b426416c93d755292d","reason":"Same source body in authored semantic page"},{"oldId":"b-8fbcacc05e2dc213f212f5d4","targetId":"b-8fbcacc05e2dc213f212f5d4","page":"vqa-image-question-controls","operation":"rewrite","beforeSHA256":"dc59f8b72c902ddc56a177ead80439b8c0443391024a1f932153c5dc856d49f3","afterSHA256":"5f663d138e257e6d2f9aa97afc7b26ce970887c62a9a53296b26becf3e7dfc3b","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-e1d7d391159984e2f7edfdb4","targetId":"b-e1d7d391159984e2f7edfdb4","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","afterSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","reason":"Same source body in authored semantic page"},{"oldId":"b-315a236f2d99f213ce016a7c","targetId":"b-315a236f2d99f213ce016a7c","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"35392381c8dab76d1ab7d15d671927c50be12cb180ab3b1faaf93e91a9cb7425","afterSHA256":"35392381c8dab76d1ab7d15d671927c50be12cb180ab3b1faaf93e91a9cb7425","reason":"Same source body in authored semantic page"},{"oldId":"b-33d6c4131249b162af03e589","targetId":"b-33d6c4131249b162af03e589","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"49712d2b5422a56bf03a9e280c5a6916614d5e65401181c538308d6d4d5f5539","afterSHA256":"49712d2b5422a56bf03a9e280c5a6916614d5e65401181c538308d6d4d5f5539","reason":"Same source body in authored semantic page"},{"oldId":"b-e0d658826b2ab2098ffd3410","targetId":"b-e0d658826b2ab2098ffd3410","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"aa3f2b08247ec91b010acd6820af5b45f343506a9021cf7091bb5eb6f7be29b5","afterSHA256":"aa3f2b08247ec91b010acd6820af5b45f343506a9021cf7091bb5eb6f7be29b5","reason":"Same source body in authored semantic page"},{"oldId":"b-c3daca48fd0739f42ef16a8b","targetId":"b-c3daca48fd0739f42ef16a8b","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"e6378866c99bf94b5778b07cba5df20b45c14b1e0d97b8c758ff5faaa9e61b5c","afterSHA256":"e6378866c99bf94b5778b07cba5df20b45c14b1e0d97b8c758ff5faaa9e61b5c","reason":"Same source body in authored semantic page"},{"oldId":"b-fea2ea0f180b40ab401f50cd","targetId":"b-fea2ea0f180b40ab401f50cd","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"d42e952fa996be83354241968ebec134716cca4b6f254ef0ba36d741e2c9d52a","afterSHA256":"d42e952fa996be83354241968ebec134716cca4b6f254ef0ba36d741e2c9d52a","reason":"Same source body in authored semantic page"},{"oldId":"b-da956c9f49669ce362c08b00","targetId":"b-da956c9f49669ce362c08b00","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"ad298559aecc4fbbef5ea5a5aa049c02bbcc3d6bc01e8ac57c78fb39a2e64c0d","afterSHA256":"ad298559aecc4fbbef5ea5a5aa049c02bbcc3d6bc01e8ac57c78fb39a2e64c0d","reason":"Same source body in authored semantic page"},{"oldId":"b-adc66098a9670562b20a7963","targetId":"b-adc66098a9670562b20a7963","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"1e4eedd320b2848fb6d1ebda6bceb80841b64b1f5aae123ecfff3e0fe78bc44b","afterSHA256":"1e4eedd320b2848fb6d1ebda6bceb80841b64b1f5aae123ecfff3e0fe78bc44b","reason":"Same source body in authored semantic page"},{"oldId":"b-883616e6e697bbf7cdc154a6","targetId":"b-883616e6e697bbf7cdc154a6","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"3415d8eb1d4cc4615ebfa98dc2e626e3b645dac5bb95ad5946934c7b15ecbe83","afterSHA256":"3415d8eb1d4cc4615ebfa98dc2e626e3b645dac5bb95ad5946934c7b15ecbe83","reason":"Same source body in authored semantic page"},{"oldId":"b-3be75c60205a263071f3f6c4","targetId":"b-3be75c60205a263071f3f6c4","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"9acda7f9b192337ee119ede76757099f4aaf432bab535c875298f12a9ced8468","afterSHA256":"9acda7f9b192337ee119ede76757099f4aaf432bab535c875298f12a9ced8468","reason":"Same source body in authored semantic page"},{"oldId":"b-89b0696fa59bf5c1926692f8","targetId":"b-89b0696fa59bf5c1926692f8","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"e528f6b5a7a16596d68d65990cac4c061b153b0cc8966498af78f8a48e645b62","afterSHA256":"e528f6b5a7a16596d68d65990cac4c061b153b0cc8966498af78f8a48e645b62","reason":"Same source body in authored semantic page"},{"oldId":"b-f22f8823f495e87f633428df","targetId":"b-f22f8823f495e87f633428df","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"0e145b5dd76bcb58b7604e77ff98b32e50b06a2fe9e9272ddab71bf6a7b6be4f","afterSHA256":"0e145b5dd76bcb58b7604e77ff98b32e50b06a2fe9e9272ddab71bf6a7b6be4f","reason":"Same source body in authored semantic page"},{"oldId":"b-f3f195949306df51df831617","targetId":"b-f3f195949306df51df831617","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"950c6c6c090f99e3582da43b47552ba52182ad69b9a7db8f3a4806ed799d878a","afterSHA256":"950c6c6c090f99e3582da43b47552ba52182ad69b9a7db8f3a4806ed799d878a","reason":"Same source body in authored semantic page"},{"oldId":"b-8774c3339dfa90a3a3cf6c5c","targetId":"b-8774c3339dfa90a3a3cf6c5c","page":"vqa-image-question-controls","operation":"move","beforeSHA256":"4ded29b4b7b1b91186b75105e032dc6dc759d25dd5b972e46524a453fcd8eeb1","afterSHA256":"4ded29b4b7b1b91186b75105e032dc6dc759d25dd5b972e46524a453fcd8eeb1","reason":"Same source body in authored semantic page"},{"oldId":"b-1d7c7e1cbc18407d8dbc3429","targetId":"b-1d7c7e1cbc18407d8dbc3429","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"492743ea7195bd5c861504a4d324708e8524f46f0d8cd7060dcebf30d3d8e7dc","afterSHA256":"492743ea7195bd5c861504a4d324708e8524f46f0d8cd7060dcebf30d3d8e7dc","reason":"Same source body in authored semantic page"},{"oldId":"b-43d45a1c6134d2096f7e9e63","targetId":"b-43d45a1c6134d2096f7e9e63","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"fcd7485dc2fc875315c5cfc8e58d8e69bd92e8ee4a2697ee9e296fe461019d35","afterSHA256":"fcd7485dc2fc875315c5cfc8e58d8e69bd92e8ee4a2697ee9e296fe461019d35","reason":"Same source body in authored semantic page"},{"oldId":"b-9a84e453f1819db01bec5e2a","targetId":"b-9a84e453f1819db01bec5e2a","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"8f89da27040af7a6e6079498b094994e2ce39ffaa7097395a9aaded0497702ee","afterSHA256":"8f89da27040af7a6e6079498b094994e2ce39ffaa7097395a9aaded0497702ee","reason":"Same source body in authored semantic page"},{"oldId":"b-ed8f1f5e80f3b077b84842e2","targetId":"b-ed8f1f5e80f3b077b84842e2","page":"vqa-lab14-structured-baseline","operation":"rewrite","beforeSHA256":"18d38de4f1210841fff977f33e9a1628d7b29c97243b63c8ab8da5b881e7ded1","afterSHA256":"bc3dba74f3009c6b434eeacf463c0a84cdb73c41e93fce898cc9e87feae52e24","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-3c5ec1a36b56b939afdd4b60","targetId":"b-3c5ec1a36b56b939afdd4b60","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"5dca656f2c377109930a783b3082cbf5d2ad432af0a58c058e98f570fe78f094","afterSHA256":"5dca656f2c377109930a783b3082cbf5d2ad432af0a58c058e98f570fe78f094","reason":"Same source body in authored semantic page"},{"oldId":"b-9f9394f004065ff5810d53b2","targetId":"b-9f9394f004065ff5810d53b2","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"dd655aba933eeca3efa133773efe24b93715ba055621e0986f10caa85cd381e9","afterSHA256":"08d0aa022e98bb75079211b073bb26e8245e159a9c4dcb17e3839b2d504f1d23","reason":"Same source body in authored semantic page"},{"oldId":"b-75f7b5b00c3788395a9958de","targetId":"b-75f7b5b00c3788395a9958de","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"7d2f78f1122d4e8c9846b7131965a7250236f51bc8d561d375ce41bac04fe7f6","afterSHA256":"7d2f78f1122d4e8c9846b7131965a7250236f51bc8d561d375ce41bac04fe7f6","reason":"Same source body in authored semantic page"},{"oldId":"b-8871753e4c0b105ccba10f3f","targetId":"b-8871753e4c0b105ccba10f3f","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","afterSHA256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","reason":"Same source body in authored semantic page"},{"oldId":"b-15dc3f404d009212c9bfcd3a","targetId":"b-15dc3f404d009212c9bfcd3a","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","afterSHA256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","reason":"Same source body in authored semantic page"},{"oldId":"b-acc8fefdc24f998df2bf1d78","targetId":"b-acc8fefdc24f998df2bf1d78","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"f7f9e17ba2ffbbaa5247457e33f0b252b802d55a7308a3bcb34e5992b15214bc","afterSHA256":"f7f9e17ba2ffbbaa5247457e33f0b252b802d55a7308a3bcb34e5992b15214bc","reason":"Same source body in authored semantic page"},{"oldId":"b-05af7803c64fef8d987fe66e","targetId":"b-05af7803c64fef8d987fe66e","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"0cf41c9762be5297e3bd4c3e1fe430529e9a1682fd90f0a815637a79f6e9baee","afterSHA256":"0cf41c9762be5297e3bd4c3e1fe430529e9a1682fd90f0a815637a79f6e9baee","reason":"Same source body in authored semantic page"},{"oldId":"b-2f9e9b7644a36f13814a3605","targetId":"b-2f9e9b7644a36f13814a3605","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","afterSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","reason":"Same source body in authored semantic page"},{"oldId":"b-b9c4b855dd7a6426aa0ed175","targetId":"b-b9c4b855dd7a6426aa0ed175","page":"vqa-lab14-structured-baseline","operation":"move","beforeSHA256":"8db070982ed999a50c85430b039bac2eb50c509c1e3f2e9d2e9fef940b72c954","afterSHA256":"8db070982ed999a50c85430b039bac2eb50c509c1e3f2e9d2e9fef940b72c954","reason":"Same source body in authored semantic page"},{"oldId":"vqa-image-question-controls-title","targetId":"b-f888f36880194c7637b03c63","operation":"retain-source-identity"},{"oldId":"vqa-lab14-structured-baseline-title","targetId":"b-2aea4816ac323a68551b3f94","operation":"retain-source-identity"},{"oldId":"vqa-lab14-structured-baseline-bridge","targetId":"b-63bf24e911528f8c423ae6d2","operation":"retain-source-identity"}],"blocks":[{"id":"b-f888f36880194c7637b03c63","type":"h2","scope":"vqa-image-question-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-f888f36880194c7637b03c63","content":"## 同图换问，再同问换图","sha256":"0bd0f754b19c72102f7a76c99b89102808d736353ca7ed52c504924980783cbb","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-274ef0b79375737f880dd447","prerequisites":[{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-274ef0b79375737f880dd447","type":"h2","scope":"explanation","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-274ef0b79375737f880dd447","content":"## 解释、算例与证据","sha256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f888f36880194c7637b03c63","nextBlock":"b-25a847d665e3181abddd827d","prerequisites":[{"id":"b-f888f36880194c7637b03c63","url":"#b-f888f36880194c7637b03c63","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-25a847d665e3181abddd827d","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-25a847d665e3181abddd827d","content":"# P54 · VQA：问题决定此刻需要哪一项信息","sha256":"19cc0322b455f4f4f375c0f9d8ca856db923c8bfb82a36655eec2226a3b67669","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-274ef0b79375737f880dd447","nextBlock":"b-9c22db528963c756bbf806ae","prerequisites":[{"id":"b-274ef0b79375737f880dd447","url":"#b-274ef0b79375737f880dd447","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-9c22db528963c756bbf806ae","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-9c22db528963c756bbf806ae","content":"问题在这里选择要读的信息，同一图可以问数量、属性和关系。本页说明沿两个方向组织检查：同图换问、同问换图。LAB14 当前已经有读取像素的可学习分支和单独训练的 question-only，对象属性字典保留为 oracle 热身；三种身份应分别阅读。","sha256":"e694c4320e7af6f266b80de8b9d0453b3a6861ef55dfd0b426416c93d755292d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-25a847d665e3181abddd827d","nextBlock":"b-8fbcacc05e2dc213f212f5d4","prerequisites":[{"id":"b-25a847d665e3181abddd827d","url":"#b-25a847d665e3181abddd827d","role":"section-title"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-8fbcacc05e2dc213f212f5d4","type":"figure","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-8fbcacc05e2dc213f212f5d4","content":"图内标签与数值：不给图片；有几个人？；需依据图像确定；谁拿着球？；需依据图像确定；球是什么颜色？；需依据图像确定；同一问题，换掉相关图像，正确答案应怎样改变？；答案为人工核对的场景属性；“不给图片”展示评价条件，不是模型问答实验。\n\n相同人物图像对应数量、持球者、球色三问；另一状态隐藏图片并将答案改为需依据图像确定。\n\n- 不给图片\n- 有几个人？\n- 需依据图像确定\n- 谁拿着球？\n- 需依据图像确定\n- 球是什么颜色？\n- 需依据图像确定\n- 同一问题，换掉相关图像，正确答案应怎样改变？\n- 答案为人工核对的场景属性；“不给图片”展示评价条件，不是模型问答实验。相同人物图像对应数量、持球者、球色三问；另一状态隐藏图片并将答案改为需依据图像确定。 [查看对应图解状态](/#/l3-r2-p54/1)[查看完整图解](/course/figures/l3-r2-p54-1.html)","sha256":"5f663d138e257e6d2f9aa97afc7b26ce970887c62a9a53296b26becf3e7dfc3b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"相同人物图像对应数量、持球者、球色三问；另一状态隐藏图片并将答案改为需依据图像确定。 查看对应图解状态","labels":["不给图片","有几个人？","需依据图像确定","谁拿着球？","需依据图像确定","球是什么颜色？","需依据图像确定","同一问题，换掉相关图像，正确答案应怎样改变？","答案为人工核对的场景属性；“不给图片”展示评价条件，不是模型问答实验。"],"description":"相同人物图像对应数量、持球者、球色三问；另一状态隐藏图片并将答案改为需依据图像确定。不给图片有几个人？需依据图像确定谁拿着球？需依据图像确定球是什么颜色？需依据图像确定同一问题，换掉相关图像，正确答案应怎样改变？答案为人工核对的场景属性；“不给图片”展示评价条件，不是模型问答实验。","evidenceIdentity":"相同人物图像对应数量、持球者、球色三问；另一状态隐藏图片并将答案改为需依据图像确定。 查看对应图解状态","images":[]},"previousBlock":"b-9c22db528963c756bbf806ae","nextBlock":"b-e1d7d391159984e2f7edfdb4","prerequisites":[{"id":"b-25a847d665e3181abddd827d","url":"#b-25a847d665e3181abddd827d","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-e1d7d391159984e2f7edfdb4","type":"h2","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-e1d7d391159984e2f7edfdb4","content":"## 课堂讲述","sha256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8fbcacc05e2dc213f212f5d4","nextBlock":"b-315a236f2d99f213ce016a7c","prerequisites":[{"id":"b-25a847d665e3181abddd827d","url":"#b-25a847d665e3181abddd827d","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-315a236f2d99f213ce016a7c","type":"p","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-315a236f2d99f213ce016a7c","content":"同一张图可以支持很多问题。问数量、颜色或者持有关系，需要从相同画面中提取不同信息。VQA 把问题本身变成输入的一部分，让任务不再只有“描述一遍”。\n训练时需要图像、问题和答案。早期系统可以从有限答案集合中选择，后来的模型也可以生成答案；两种接口不同，但都要检查回答是否真正利用了图像。\n有些问题可以靠常见答案猜对。例如某种训练分布里，某个问法总对应同一种颜色。为了区别猜测和视觉利用，可以保持问题不变、替换相关图像，再看答案是否合理改变。我们先看清成功应该意味着什么，而不是只看一次回答听起来像不像人。","sha256":"35392381c8dab76d1ab7d15d671927c50be12cb180ab3b1faaf93e91a9cb7425","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e1d7d391159984e2f7edfdb4","nextBlock":"b-33d6c4131249b162af03e589","prerequisites":[{"id":"b-e1d7d391159984e2f7edfdb4","url":"#b-e1d7d391159984e2f7edfdb4","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-33d6c4131249b162af03e589","type":"h3","scope":"vqa-question-selects-information","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-33d6c4131249b162af03e589","content":"### VQA：同图换问题，同问换图","sha256":"49712d2b5422a56bf03a9e280c5a6916614d5e65401181c538308d6d4d5f5539","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"VQA：同图换问题，同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-question-selects-information","formulas":[],"previousBlock":"b-315a236f2d99f213ce016a7c","nextBlock":"b-e0d658826b2ab2098ffd3410","prerequisites":[{"id":"b-e1d7d391159984e2f7edfdb4","url":"#b-e1d7d391159984e2f7edfdb4","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"},{"id":"b-e0d658826b2ab2098ffd3410","url":"#b-e0d658826b2ab2098ffd3410","relation":"same-reasoning"},{"id":"b-c3daca48fd0739f42ef16a8b","url":"#b-c3daca48fd0739f42ef16a8b","relation":"same-reasoning"},{"id":"b-fea2ea0f180b40ab401f50cd","url":"#b-fea2ea0f180b40ab401f50cd","relation":"same-reasoning"},{"id":"b-da956c9f49669ce362c08b00","url":"#b-da956c9f49669ce362c08b00","relation":"same-reasoning"}]},{"id":"b-e0d658826b2ab2098ffd3410","type":"p","scope":"vqa-question-selects-information","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-e0d658826b2ab2098ffd3410","content":"VQA 的目标是 $p(a\\mid I,q)$。问题 $q$ 不只是输出旁的装饰，而是决定应该从图像中读取什么：问“有几个人”需要数量，问“谁拿球”需要关系，问“球是什么颜色”需要属性。保持图像固定，改变问题，正确答案就可能改变；保持问题固定，改变相关图像，正确答案也应变化。","sha256":"aa3f2b08247ec91b010acd6820af5b45f343506a9021cf7091bb5eb6f7be29b5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"VQA：同图换问题，同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-question-selects-information","formulas":[{"latex":"p(a\\mid I,q)","definitionContext":"VQA 的目标是 $p(a\\mid I,q)$。问题 $q$ 不只是输出旁的装饰，而是决定应该从图像中读取什么：问“有几个人”需要数量，问“谁拿球”需要关系，问“球是什么颜色”需要属性。保持图像固定，改变问题，正确答案就可能改变；保持问题固定，改变相关图像，正确答案也应变化。\n一个封闭答案模型可以在预设词表里分类，一个生成式模型可以逐词回答。前者易于定义准确率，但答案范围受词表限制；后者表达更灵活，却需要处理同义表达、冗长回答和证据对应。不能只把分类头输出的一个标签换成长句，就称为具有新的视觉推理能力。\n语言先验是有用的知识，也可能成为捷径。如果训练里“左边物体什么颜色”通常答红色，question-only 模型能取得不错成绩，却无法应对同问题下的蓝物体。VQA v2 的研究用相同问题、不同正确答案的成对图像，加强视觉证据在评价中的作用。这种设计把“答对”进一步变成“答对所需的信息确实来自图像”。\n课堂上可先固定图 A 左红右蓝：问左颜色答红，问右颜色答蓝。再换成左蓝右红的图 B，保持左颜色问题，此时应答蓝。三次比较让图像条件和问题条件分别显示作用。读取场景字典中的 `left_color` 是已知属性 oracle，它能检验问答逻辑，但不能检验从像素感知颜色。"},{"latex":"q","definitionContext":"VQA 的目标是 $p(a\\mid I,q)$。问题 $q$ 不只是输出旁的装饰，而是决定应该从图像中读取什么：问“有几个人”需要数量，问“谁拿球”需要关系，问“球是什么颜色”需要属性。保持图像固定，改变问题，正确答案就可能改变；保持问题固定，改变相关图像，正确答案也应变化。\n一个封闭答案模型可以在预设词表里分类，一个生成式模型可以逐词回答。前者易于定义准确率，但答案范围受词表限制；后者表达更灵活，却需要处理同义表达、冗长回答和证据对应。不能只把分类头输出的一个标签换成长句，就称为具有新的视觉推理能力。\n语言先验是有用的知识，也可能成为捷径。如果训练里“左边物体什么颜色”通常答红色，question-only 模型能取得不错成绩，却无法应对同问题下的蓝物体。VQA v2 的研究用相同问题、不同正确答案的成对图像，加强视觉证据在评价中的作用。这种设计把“答对”进一步变成“答对所需的信息确实来自图像”。\n课堂上可先固定图 A 左红右蓝：问左颜色答红，问右颜色答蓝。再换成左蓝右红的图 B，保持左颜色问题，此时应答蓝。三次比较让图像条件和问题条件分别显示作用。读取场景字典中的 `left_color` 是已知属性 oracle，它能检验问答逻辑，但不能检验从像素感知颜色。"}],"previousBlock":"b-33d6c4131249b162af03e589","nextBlock":"b-c3daca48fd0739f42ef16a8b","prerequisites":[{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"},{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","relation":"same-reasoning"},{"id":"b-c3daca48fd0739f42ef16a8b","url":"#b-c3daca48fd0739f42ef16a8b","relation":"same-reasoning"},{"id":"b-fea2ea0f180b40ab401f50cd","url":"#b-fea2ea0f180b40ab401f50cd","relation":"same-reasoning"},{"id":"b-da956c9f49669ce362c08b00","url":"#b-da956c9f49669ce362c08b00","relation":"same-reasoning"}]},{"id":"b-c3daca48fd0739f42ef16a8b","type":"p","scope":"vqa-question-selects-information","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-c3daca48fd0739f42ef16a8b","content":"一个封闭答案模型可以在预设词表里分类，一个生成式模型可以逐词回答。前者易于定义准确率，但答案范围受词表限制；后者表达更灵活，却需要处理同义表达、冗长回答和证据对应。不能只把分类头输出的一个标签换成长句，就称为具有新的视觉推理能力。","sha256":"e6378866c99bf94b5778b07cba5df20b45c14b1e0d97b8c758ff5faaa9e61b5c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"VQA：同图换问题，同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-question-selects-information","formulas":[],"previousBlock":"b-e0d658826b2ab2098ffd3410","nextBlock":"b-fea2ea0f180b40ab401f50cd","prerequisites":[{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"},{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","relation":"same-reasoning"},{"id":"b-e0d658826b2ab2098ffd3410","url":"#b-e0d658826b2ab2098ffd3410","relation":"same-reasoning"},{"id":"b-fea2ea0f180b40ab401f50cd","url":"#b-fea2ea0f180b40ab401f50cd","relation":"same-reasoning"},{"id":"b-da956c9f49669ce362c08b00","url":"#b-da956c9f49669ce362c08b00","relation":"same-reasoning"}]},{"id":"b-fea2ea0f180b40ab401f50cd","type":"p","scope":"vqa-question-selects-information","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-fea2ea0f180b40ab401f50cd","content":"语言先验是有用的知识，也可能成为捷径。如果训练里“左边物体什么颜色”通常答红色，question-only 模型能取得不错成绩，却无法应对同问题下的蓝物体。VQA v2 的研究用相同问题、不同正确答案的成对图像，加强视觉证据在评价中的作用。这种设计把“答对”进一步变成“答对所需的信息确实来自图像”。","sha256":"d42e952fa996be83354241968ebec134716cca4b6f254ef0ba36d741e2c9d52a","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"VQA：同图换问题，同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-question-selects-information","formulas":[],"previousBlock":"b-c3daca48fd0739f42ef16a8b","nextBlock":"b-da956c9f49669ce362c08b00","prerequisites":[{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"},{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","relation":"same-reasoning"},{"id":"b-e0d658826b2ab2098ffd3410","url":"#b-e0d658826b2ab2098ffd3410","relation":"same-reasoning"},{"id":"b-c3daca48fd0739f42ef16a8b","url":"#b-c3daca48fd0739f42ef16a8b","relation":"same-reasoning"},{"id":"b-da956c9f49669ce362c08b00","url":"#b-da956c9f49669ce362c08b00","relation":"same-reasoning"}]},{"id":"b-da956c9f49669ce362c08b00","type":"p","scope":"vqa-question-selects-information","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-da956c9f49669ce362c08b00","content":"课堂上可先固定图 A 左红右蓝：问左颜色答红，问右颜色答蓝。再换成左蓝右红的图 B，保持左颜色问题，此时应答蓝。三次比较让图像条件和问题条件分别显示作用。读取场景字典中的 `left_color` 是已知属性 oracle，它能检验问答逻辑，但不能检验从像素感知颜色。","sha256":"ad298559aecc4fbbef5ea5a5aa049c02bbcc3d6bc01e8ac57c78fb39a2e64c0d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"VQA：同图换问题，同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-question-selects-information","formulas":[],"previousBlock":"b-fea2ea0f180b40ab401f50cd","nextBlock":"b-cddcb514f5a3dcf0af1a88f7","prerequisites":[{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"},{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","relation":"same-reasoning"},{"id":"b-e0d658826b2ab2098ffd3410","url":"#b-e0d658826b2ab2098ffd3410","relation":"same-reasoning"},{"id":"b-c3daca48fd0739f42ef16a8b","url":"#b-c3daca48fd0739f42ef16a8b","relation":"same-reasoning"},{"id":"b-fea2ea0f180b40ab401f50cd","url":"#b-fea2ea0f180b40ab401f50cd","relation":"same-reasoning"}]},{"id":"b-cddcb514f5a3dcf0af1a88f7","type":"h4","scope":"vqa-image-question-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-cddcb514f5a3dcf0af1a88f7","content":"#### 图示条件与公式约定","sha256":"ef7f915c1ae63ed67bd23fecf050249273ba0efa8965d2c8d1c1c6b2d3178122","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-da956c9f49669ce362c08b00","nextBlock":"b-adc66098a9670562b20a7963","prerequisites":[{"id":"b-33d6c4131249b162af03e589","url":"#b-33d6c4131249b162af03e589","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-adc66098a9670562b20a7963","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-adc66098a9670562b20a7963","content":"图示答案是人工核对的场景属性：2 人、红衣人持球、橙色篮球。无图状态不展示模型猜测，没有执行新 VQA 实验。","sha256":"1e4eedd320b2848fb6d1ebda6bceb80841b64b1f5aae123ecfff3e0fe78bc44b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-cddcb514f5a3dcf0af1a88f7","nextBlock":"b-72d21bf5376adf1e503f67fe","prerequisites":[{"id":"b-cddcb514f5a3dcf0af1a88f7","url":"#b-cddcb514f5a3dcf0af1a88f7","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-72d21bf5376adf1e503f67fe","type":"h4","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-72d21bf5376adf1e503f67fe","content":"#### 这条线索的展开与后续阅读","sha256":"48b8229f7559b36ed4d5628416e3991e5c73293fbce43374db67386c8736f3bc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-adc66098a9670562b20a7963","nextBlock":"b-883616e6e697bbf7cdc154a6","prerequisites":[{"id":"b-cddcb514f5a3dcf0af1a88f7","url":"#b-cddcb514f5a3dcf0af1a88f7","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-883616e6e697bbf7cdc154a6","type":"ul","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-883616e6e697bbf7cdc154a6","content":"- [问题指定视觉读取，也可能泄露答案先验｜问题从哪里出现](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-problem/0) 本页说明\n- [问题指定视觉读取，也可能泄露答案先验｜核心改动怎样工作](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-mechanism/0) 本页说明\n- [问题指定视觉读取，也可能泄露答案先验｜一起算一个完整例子](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-example/0) 本页说明\n- [问题指定视觉读取，也可能泄露答案先验｜怎样检查这项解释](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-evidence/0) 本页说明\n- [问题指定视觉读取，也可能泄露答案先验｜什么结论还不能推出](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-boundary/0) 本页说明","sha256":"3415d8eb1d4cc4615ebfa98dc2e626e3b645dac5bb95ad5946934c7b15ecbe83","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-72d21bf5376adf1e503f67fe","nextBlock":"b-3be75c60205a263071f3f6c4","prerequisites":[{"id":"b-72d21bf5376adf1e503f67fe","url":"#b-72d21bf5376adf1e503f67fe","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-3be75c60205a263071f3f6c4","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-3be75c60205a263071f3f6c4","content":"- [问题指定视觉读取，也可能泄露答案先验｜问题从哪里出现](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-problem/0) 本页说明","sha256":"9acda7f9b192337ee119ede76757099f4aaf432bab535c875298f12a9ced8468","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-883616e6e697bbf7cdc154a6","nextBlock":"b-89b0696fa59bf5c1926692f8","prerequisites":[{"id":"b-72d21bf5376adf1e503f67fe","url":"#b-72d21bf5376adf1e503f67fe","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-89b0696fa59bf5c1926692f8","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-89b0696fa59bf5c1926692f8","content":"- [问题指定视觉读取，也可能泄露答案先验｜核心改动怎样工作](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-mechanism/0) 本页说明","sha256":"e528f6b5a7a16596d68d65990cac4c061b153b0cc8966498af78f8a48e645b62","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3be75c60205a263071f3f6c4","nextBlock":"b-f22f8823f495e87f633428df","prerequisites":[{"id":"b-72d21bf5376adf1e503f67fe","url":"#b-72d21bf5376adf1e503f67fe","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-f22f8823f495e87f633428df","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-f22f8823f495e87f633428df","content":"- [问题指定视觉读取，也可能泄露答案先验｜一起算一个完整例子](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-example/0) 本页说明","sha256":"0e145b5dd76bcb58b7604e77ff98b32e50b06a2fe9e9272ddab71bf6a7b6be4f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-89b0696fa59bf5c1926692f8","nextBlock":"b-f3f195949306df51df831617","prerequisites":[{"id":"b-72d21bf5376adf1e503f67fe","url":"#b-72d21bf5376adf1e503f67fe","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-f3f195949306df51df831617","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-f3f195949306df51df831617","content":"- [问题指定视觉读取，也可能泄露答案先验｜怎样检查这项解释](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-evidence/0) 本页说明","sha256":"950c6c6c090f99e3582da43b47552ba52182ad69b9a7db8f3a4806ed799d878a","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f22f8823f495e87f633428df","nextBlock":"b-8774c3339dfa90a3a3cf6c5c","prerequisites":[{"id":"b-72d21bf5376adf1e503f67fe","url":"#b-72d21bf5376adf1e503f67fe","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-8774c3339dfa90a3a3cf6c5c","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-8774c3339dfa90a3a3cf6c5c","content":"- [问题指定视觉读取，也可能泄露答案先验｜什么结论还不能推出](/supplement.html?return=l3-r2-p54/0#/l3-vqa-baseline-boundary/0) 本页说明","sha256":"4ded29b4b7b1b91186b75105e032dc6dc759d25dd5b972e46524a453fcd8eeb1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-image-question-controls","title":"同图换问，再同问换图","number":1},"sectionTitle":"同图换问，再同问换图","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f3f195949306df51df831617","nextBlock":null,"prerequisites":[{"id":"b-72d21bf5376adf1e503f67fe","url":"#b-72d21bf5376adf1e503f67fe","role":"section-title"},{"id":"b-9c22db528963c756bbf806ae","url":"#b-9c22db528963c756bbf806ae","role":"problem-context"}],"relatedBlocks":[{"id":"b-8fbcacc05e2dc213f212f5d4","url":"#b-8fbcacc05e2dc213f212f5d4","relation":"figure-and-explanation"}]},{"id":"b-2aea4816ac323a68551b3f94","type":"h2","scope":"vqa-lab14-structured-baseline","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-2aea4816ac323a68551b3f94","content":"## 先算结构化基线，再读像素问答结果","sha256":"265a879733a182940c0fb549a183da068a7f9842c99faded493dca2e8ce8fd19","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-63bf24e911528f8c423ae6d2","prerequisites":[{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-63bf24e911528f8c423ae6d2","type":"p","scope":"vqa-lab14-structured-baseline","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-63bf24e911528f8c423ae6d2","content":"一次无图答对不足以判断图像有没有作用；先根据三个任务计算可比较的基线。","sha256":"f1837721665036b669ff63ae0cb25923c3b9cd8c5b6f2f8b6d8b8b20cf3b221a","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2aea4816ac323a68551b3f94","nextBlock":"b-1d7c7e1cbc18407d8dbc3429","prerequisites":[{"id":"b-2aea4816ac323a68551b3f94","url":"#b-2aea4816ac323a68551b3f94","role":"section-title"}],"relatedBlocks":[]},{"id":"b-1d7c7e1cbc18407d8dbc3429","type":"h3","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-1d7c7e1cbc18407d8dbc3429","content":"### 已保存的像素 VQA：为什么 44.44% 是结构化基线","sha256":"492743ea7195bd5c861504a4d324708e8524f46f0d8cd7060dcebf30d3d8e7dc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[],"previousBlock":"b-63bf24e911528f8c423ae6d2","nextBlock":"b-43d45a1c6134d2096f7e9e63","prerequisites":[{"id":"b-2aea4816ac323a68551b3f94","url":"#b-2aea4816ac323a68551b3f94","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-43d45a1c6134d2096f7e9e63","url":"#b-43d45a1c6134d2096f7e9e63","relation":"same-reasoning"},{"id":"b-9a84e453f1819db01bec5e2a","url":"#b-9a84e453f1819db01bec5e2a","relation":"same-reasoning"},{"id":"b-ed8f1f5e80f3b077b84842e2","url":"#b-ed8f1f5e80f3b077b84842e2","relation":"same-reasoning"},{"id":"b-3c5ec1a36b56b939afdd4b60","url":"#b-3c5ec1a36b56b939afdd4b60","relation":"same-reasoning"},{"id":"b-9f9394f004065ff5810d53b2","url":"#b-9f9394f004065ff5810d53b2","relation":"same-reasoning"},{"id":"b-75f7b5b00c3788395a9958de","url":"#b-75f7b5b00c3788395a9958de","relation":"same-reasoning"}]},{"id":"b-43d45a1c6134d2096f7e9e63","type":"p","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-43d45a1c6134d2096f7e9e63","content":"LAB14 在 2026-09-22 保存了本地 CPU 学习结果，本次整理没有重跑。720 张训练图和 180 张保留图先按图像生成和划分，再对每图提出三种固定模板问题，共 2160/540 个图问对。模型实际读取 RGB 像素和问题 ID，不读取生成器属性字典。","sha256":"fcd7485dc2fc875315c5cfc8e58d8e69bd92e8ee4a2697ee9e296fe461019d35","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[],"previousBlock":"b-1d7c7e1cbc18407d8dbc3429","nextBlock":"b-9a84e453f1819db01bec5e2a","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","relation":"same-reasoning"},{"id":"b-9a84e453f1819db01bec5e2a","url":"#b-9a84e453f1819db01bec5e2a","relation":"same-reasoning"},{"id":"b-ed8f1f5e80f3b077b84842e2","url":"#b-ed8f1f5e80f3b077b84842e2","relation":"same-reasoning"},{"id":"b-3c5ec1a36b56b939afdd4b60","url":"#b-3c5ec1a36b56b939afdd4b60","relation":"same-reasoning"},{"id":"b-9f9394f004065ff5810d53b2","url":"#b-9f9394f004065ff5810d53b2","relation":"same-reasoning"},{"id":"b-75f7b5b00c3788395a9958de","url":"#b-75f7b5b00c3788395a9958de","relation":"same-reasoning"}]},{"id":"b-9a84e453f1819db01bec5e2a","type":"p","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-9a84e453f1819db01bec5e2a","content":"保留任务分别是数量二选一、左侧颜色三选一、右侧形状二选一。只知道问题时，各任务的最佳常量正确率为 $1/2,1/3,1/2$，平均 $(1/2+1/3+1/2)/3=4/9$。把所有七种答案混在一起写“机会率 1/7”会忽略问题已给出的答案类型。","sha256":"8f89da27040af7a6e6079498b094994e2ce39ffaa7097395a9aaded0497702ee","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[{"latex":"1/2,1/3,1/2","definitionContext":"LAB14 在 2026-09-22 保存了本地 CPU 学习结果，本次整理没有重跑。720 张训练图和 180 张保留图先按图像生成和划分，再对每图提出三种固定模板问题，共 2160/540 个图问对。模型实际读取 RGB 像素和问题 ID，不读取生成器属性字典。\n保留任务分别是数量二选一、左侧颜色三选一、右侧形状二选一。只知道问题时，各任务的最佳常量正确率为 $1/2,1/3,1/2$，平均 $(1/2+1/3+1/2)/3=4/9$。把所有七种答案混在一起写“机会率 1/7”会忽略问题已给出的答案类型。\n最后一行不是随机机会率：数据按组合排列，循环移位常保留数量却恰好改变形状，因此总分相同仍有不同错误来源。正确图像下的模型通过了三个有限任务；像素置零和错配的下降支持它使用正确视觉输入。训练左侧红色比例为 73.61%，保留颜色均衡，进一步显示问题先验不能替代视觉读取。\n![同图换问、同问换图](/assets/display/2f7b2168f5b21ee2095709fa.webp)\n问题输入在这里是三个模板 ID，不是任意句子的语言理解。先承认这个小实验确实完成的像素问答，再把开放问题、未见对象与组合列为下一层检验。"},{"latex":"(1/2+1/3+1/2)/3=4/9","definitionContext":"LAB14 在 2026-09-22 保存了本地 CPU 学习结果，本次整理没有重跑。720 张训练图和 180 张保留图先按图像生成和划分，再对每图提出三种固定模板问题，共 2160/540 个图问对。模型实际读取 RGB 像素和问题 ID，不读取生成器属性字典。\n保留任务分别是数量二选一、左侧颜色三选一、右侧形状二选一。只知道问题时，各任务的最佳常量正确率为 $1/2,1/3,1/2$，平均 $(1/2+1/3+1/2)/3=4/9$。把所有七种答案混在一起写“机会率 1/7”会忽略问题已给出的答案类型。\n最后一行不是随机机会率：数据按组合排列，循环移位常保留数量却恰好改变形状，因此总分相同仍有不同错误来源。正确图像下的模型通过了三个有限任务；像素置零和错配的下降支持它使用正确视觉输入。训练左侧红色比例为 73.61%，保留颜色均衡，进一步显示问题先验不能替代视觉读取。\n![同图换问、同问换图](/assets/display/2f7b2168f5b21ee2095709fa.webp)\n问题输入在这里是三个模板 ID，不是任意句子的语言理解。先承认这个小实验确实完成的像素问答，再把开放问题、未见对象与组合列为下一层检验。"}],"previousBlock":"b-43d45a1c6134d2096f7e9e63","nextBlock":"b-ed8f1f5e80f3b077b84842e2","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","relation":"same-reasoning"},{"id":"b-43d45a1c6134d2096f7e9e63","url":"#b-43d45a1c6134d2096f7e9e63","relation":"same-reasoning"},{"id":"b-ed8f1f5e80f3b077b84842e2","url":"#b-ed8f1f5e80f3b077b84842e2","relation":"same-reasoning"},{"id":"b-3c5ec1a36b56b939afdd4b60","url":"#b-3c5ec1a36b56b939afdd4b60","relation":"same-reasoning"},{"id":"b-9f9394f004065ff5810d53b2","url":"#b-9f9394f004065ff5810d53b2","relation":"same-reasoning"},{"id":"b-75f7b5b00c3788395a9958de","url":"#b-75f7b5b00c3788395a9958de","relation":"same-reasoning"}]},{"id":"b-ed8f1f5e80f3b077b84842e2","type":"table","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-ed8f1f5e80f3b077b84842e2","content":"| 保留条件 | 数量 | 左色 | 右形 | 全部 |\n| --- | --- | --- | --- | --- |\n| 图像＋问题 | 100% | 100% | 100% | 100% |\n| 单独训练 question-only | 50% | 33.33% | 50% | 44.44% |\n| 同一视觉模型，像素置零 | 50% | 33.33% | 50% | 44.44% |\n| 同一视觉模型，循环错配图像 | 83.33% | 50% | 0% | 44.44% |","sha256":"bc3dba74f3009c6b434eeacf463c0a84cdb73c41e93fce898cc9e87feae52e24","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[],"table":{"headers":["保留条件","数量","左色","右形","全部"],"rows":[["保留条件","数量","左色","右形","全部"],["图像＋问题","100%","100%","100%","100%"],["单独训练 question-only","50%","33.33%","50%","44.44%"],["同一视觉模型，像素置零","50%","33.33%","50%","44.44%"],["同一视觉模型，循环错配图像","83.33%","50%","0%","44.44%"]]},"previousBlock":"b-9a84e453f1819db01bec5e2a","nextBlock":"b-3c5ec1a36b56b939afdd4b60","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","relation":"same-reasoning"},{"id":"b-43d45a1c6134d2096f7e9e63","url":"#b-43d45a1c6134d2096f7e9e63","relation":"same-reasoning"},{"id":"b-9a84e453f1819db01bec5e2a","url":"#b-9a84e453f1819db01bec5e2a","relation":"same-reasoning"},{"id":"b-3c5ec1a36b56b939afdd4b60","url":"#b-3c5ec1a36b56b939afdd4b60","relation":"same-reasoning"},{"id":"b-9f9394f004065ff5810d53b2","url":"#b-9f9394f004065ff5810d53b2","relation":"same-reasoning"},{"id":"b-75f7b5b00c3788395a9958de","url":"#b-75f7b5b00c3788395a9958de","relation":"same-reasoning"}]},{"id":"b-3c5ec1a36b56b939afdd4b60","type":"p","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-3c5ec1a36b56b939afdd4b60","content":"最后一行不是随机机会率：数据按组合排列，循环移位常保留数量却恰好改变形状，因此总分相同仍有不同错误来源。正确图像下的模型通过了三个有限任务；像素置零和错配的下降支持它使用正确视觉输入。训练左侧红色比例为 73.61%，保留颜色均衡，进一步显示问题先验不能替代视觉读取。","sha256":"5dca656f2c377109930a783b3082cbf5d2ad432af0a58c058e98f570fe78f094","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[],"previousBlock":"b-ed8f1f5e80f3b077b84842e2","nextBlock":"b-9f9394f004065ff5810d53b2","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","relation":"same-reasoning"},{"id":"b-43d45a1c6134d2096f7e9e63","url":"#b-43d45a1c6134d2096f7e9e63","relation":"same-reasoning"},{"id":"b-9a84e453f1819db01bec5e2a","url":"#b-9a84e453f1819db01bec5e2a","relation":"same-reasoning"},{"id":"b-ed8f1f5e80f3b077b84842e2","url":"#b-ed8f1f5e80f3b077b84842e2","relation":"same-reasoning"},{"id":"b-9f9394f004065ff5810d53b2","url":"#b-9f9394f004065ff5810d53b2","relation":"same-reasoning"},{"id":"b-75f7b5b00c3788395a9958de","url":"#b-75f7b5b00c3788395a9958de","relation":"same-reasoning"}]},{"id":"b-9f9394f004065ff5810d53b2","type":"p","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-9f9394f004065ff5810d53b2","content":"![同图换问、同问换图](/assets/display/2f7b2168f5b21ee2095709fa.webp)","sha256":"08d0aa022e98bb75079211b073bb26e8245e159a9c4dcb17e3839b2d504f1d23","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[],"previousBlock":"b-3c5ec1a36b56b939afdd4b60","nextBlock":"b-75f7b5b00c3788395a9958de","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","relation":"same-reasoning"},{"id":"b-43d45a1c6134d2096f7e9e63","url":"#b-43d45a1c6134d2096f7e9e63","relation":"same-reasoning"},{"id":"b-9a84e453f1819db01bec5e2a","url":"#b-9a84e453f1819db01bec5e2a","relation":"same-reasoning"},{"id":"b-ed8f1f5e80f3b077b84842e2","url":"#b-ed8f1f5e80f3b077b84842e2","relation":"same-reasoning"},{"id":"b-3c5ec1a36b56b939afdd4b60","url":"#b-3c5ec1a36b56b939afdd4b60","relation":"same-reasoning"},{"id":"b-75f7b5b00c3788395a9958de","url":"#b-75f7b5b00c3788395a9958de","relation":"same-reasoning"}]},{"id":"b-75f7b5b00c3788395a9958de","type":"p","scope":"vqa-pixel-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-75f7b5b00c3788395a9958de","content":"问题输入在这里是三个模板 ID，不是任意句子的语言理解。先承认这个小实验确实完成的像素问答，再把开放问题、未见对象与组合列为下一层检验。","sha256":"7d2f78f1122d4e8c9846b7131965a7250236f51bc8d561d375ce41bac04fe7f6","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"已保存的像素 VQA：为什么 44.44% 是结构化基线","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":"vqa-pixel-evidence","formulas":[],"previousBlock":"b-9f9394f004065ff5810d53b2","nextBlock":"b-8871753e4c0b105ccba10f3f","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","relation":"same-reasoning"},{"id":"b-43d45a1c6134d2096f7e9e63","url":"#b-43d45a1c6134d2096f7e9e63","relation":"same-reasoning"},{"id":"b-9a84e453f1819db01bec5e2a","url":"#b-9a84e453f1819db01bec5e2a","relation":"same-reasoning"},{"id":"b-ed8f1f5e80f3b077b84842e2","url":"#b-ed8f1f5e80f3b077b84842e2","relation":"same-reasoning"},{"id":"b-3c5ec1a36b56b939afdd4b60","url":"#b-3c5ec1a36b56b939afdd4b60","relation":"same-reasoning"},{"id":"b-9f9394f004065ff5810d53b2","url":"#b-9f9394f004065ff5810d53b2","relation":"same-reasoning"}]},{"id":"b-8871753e4c0b105ccba10f3f","type":"h3","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-8871753e4c0b105ccba10f3f","content":"### 沿着图解追问","sha256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-75f7b5b00c3788395a9958de","nextBlock":"b-15dc3f404d009212c9bfcd3a","prerequisites":[{"id":"b-1d7c7e1cbc18407d8dbc3429","url":"#b-1d7c7e1cbc18407d8dbc3429","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-15dc3f404d009212c9bfcd3a","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-15dc3f404d009212c9bfcd3a","content":"以下是备课用的模拟师生对话，不是课堂实录。","sha256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8871753e4c0b105ccba10f3f","nextBlock":"b-acc8fefdc24f998df2bf1d78","prerequisites":[{"id":"b-8871753e4c0b105ccba10f3f","url":"#b-8871753e4c0b105ccba10f3f","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-acc8fefdc24f998df2bf1d78","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-acc8fefdc24f998df2bf1d78","content":"**学生：**问题模型也能答对 44%，说明图像不重要吗？","sha256":"f7f9e17ba2ffbbaa5247457e33f0b252b802d55a7308a3bcb34e5992b15214bc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-15dc3f404d009212c9bfcd3a","nextBlock":"b-05af7803c64fef8d987fe66e","prerequisites":[{"id":"b-8871753e4c0b105ccba10f3f","url":"#b-8871753e4c0b105ccba10f3f","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-05af7803c64fef8d987fe66e","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-05af7803c64fef8d987fe66e","content":"**教师：**先看任务结构：二选一、三选一、二选一的常量基线平均就是 4/9。视觉模型在相同保留图问对达到 100%，且错配后下降，才说明这里增加的图像条件有作用。","sha256":"0cf41c9762be5297e3bd4c3e1fe430529e9a1682fd90f0a815637a79f6e9baee","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-acc8fefdc24f998df2bf1d78","nextBlock":"b-2f9e9b7644a36f13814a3605","prerequisites":[{"id":"b-8871753e4c0b105ccba10f3f","url":"#b-8871753e4c0b105ccba10f3f","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2f9e9b7644a36f13814a3605","type":"h2","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-2f9e9b7644a36f13814a3605","content":"## 沿这个问题继续","sha256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-05af7803c64fef8d987fe66e","nextBlock":"b-b9c4b855dd7a6426aa0ed175","prerequisites":[{"id":"b-8871753e4c0b105ccba10f3f","url":"#b-8871753e4c0b105ccba10f3f","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b9c4b855dd7a6426aa0ed175","type":"p","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p54.html#b-b9c4b855dd7a6426aa0ed175","content":"先回看本页画面并指出输入、条件和输出，再按“VQA：同图换问题，同问换图 → 已保存的像素 VQA：为什么 44.44% 是结构化基线”阅读。相关实验以其已保存结果为起点，需要动手时再顺序执行 Notebook；","sha256":"8db070982ed999a50c85430b039bac2eb50c509c1e3f2e9d2e9fef940b72c954","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"vqa-lab14-structured-baseline","title":"先算结构化基线，再读像素问答结果","number":2},"sectionTitle":"先算结构化基线，再读像素问答结果","currentQuestion":"局部对应、问题条件和联合预训练分别增加了哪条关系？","sourceSlides":["l3-r2-p54"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2f9e9b7644a36f13814a3605","nextBlock":null,"prerequisites":[{"id":"b-2f9e9b7644a36f13814a3605","url":"#b-2f9e9b7644a36f13814a3605","role":"section-title"},{"id":"b-63bf24e911528f8c423ae6d2","url":"#b-63bf24e911528f8c423ae6d2","role":"problem-context"}],"relatedBlocks":[]}]}