{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"CLS 09 · 图像条件输出：描述与问答","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html","markdown":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.md","pages":[{"id":"reading-opening","title":"CLS 09 · 图像条件输出：描述与问答","unit":"CLS 09 · 图像条件输出：描述与问答","unitId":"cls10-图像描述解码与视觉问答","chapter":"course","number":1},{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","unit":"CLS 09 · 图像条件输出：描述与问答","unitId":"cls10-图像描述解码与视觉问答","chapter":"course","number":2},{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","unit":"CLS 09 · 图像条件输出：描述与问答","unitId":"cls10-图像描述解码与视觉问答","chapter":"course","number":3},{"id":"reading-block-0bf24433979946a493db9110cc38c355","title":"看清图像条件，再打开解码器","unit":"CLS 09 · 图像条件输出：描述与问答","unitId":"cls10-图像描述解码与视觉问答","chapter":"course","number":4}],"aliases":[{"oldId":"b-f04e0f6f56c9ce790f4c9dae","targetId":"b-f04e0f6f56c9ce790f4c9dae","page":"reading-opening","operation":"move","beforeSHA256":"0869a21a5e70ead026f87cc0ba7340e1480c2d98d71496705a4271b274042716","afterSHA256":"0869a21a5e70ead026f87cc0ba7340e1480c2d98d71496705a4271b274042716","reason":"Same source body in authored semantic page"},{"oldId":"b-86a8da8bed3881dcda2eff01","targetId":"b-86a8da8bed3881dcda2eff01","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-a6908b3621fbebcb832caf23","targetId":"b-a6908b3621fbebcb832caf23","page":"reading-opening","operation":"move","beforeSHA256":"a192bf58b03c510999643a276b8d2261f6c9389ea09aebd6ad3419fe70e2a7b9","afterSHA256":"a192bf58b03c510999643a276b8d2261f6c9389ea09aebd6ad3419fe70e2a7b9","reason":"Same source body in authored semantic page"},{"oldId":"b-537c165a996846e1f65ebcd9","targetId":"b-537c165a996846e1f65ebcd9","page":"reading-opening","operation":"move","beforeSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","afterSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","reason":"Same source body in authored semantic page"},{"oldId":"b-c8c22e4cb390d063c7e1f222","targetId":"b-c8c22e4cb390d063c7e1f222","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","afterSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","reason":"Same source body in authored semantic page"},{"oldId":"b-a013754450e2144dbeee4353","targetId":"b-a013754450e2144dbeee4353","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"6b3101dc74d613fd527b60ff21d1ac65db81cca6fc49d581090c9a90654d2057","afterSHA256":"6b3101dc74d613fd527b60ff21d1ac65db81cca6fc49d581090c9a90654d2057","reason":"Same source body in authored semantic page"},{"oldId":"b-c2e7548ea1060e22a275417f","targetId":"b-c2e7548ea1060e22a275417f","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"98588b7069ba7f92211a8e6cd5c8be5f8249ccf2425e964a426643a339f7a008","afterSHA256":"98588b7069ba7f92211a8e6cd5c8be5f8249ccf2425e964a426643a339f7a008","reason":"Same source body in authored semantic page"},{"oldId":"b-6f20cc81c7ddad3d737ea63d","targetId":"b-6f20cc81c7ddad3d737ea63d","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"46d053d1cc569605d484255e6af2c3466f7d40f4fe2b28cec955d4de4a9dd9d8","afterSHA256":"46d053d1cc569605d484255e6af2c3466f7d40f4fe2b28cec955d4de4a9dd9d8","reason":"Same source body in authored semantic page"},{"oldId":"b-b55d2fb1392859a5bf2b3eea","targetId":"b-b55d2fb1392859a5bf2b3eea","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"5d04e11d8c2e016f93a6fd003c5aafbcfa45431a430ae351461bb69f38e77a4d","afterSHA256":"5d04e11d8c2e016f93a6fd003c5aafbcfa45431a430ae351461bb69f38e77a4d","reason":"Same source body in authored semantic page"},{"oldId":"b-2a68c81d4c275a336fa14035","targetId":"b-2a68c81d4c275a336fa14035","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"621909530fe228f941698b8aa1acd5e19c591377be6a124d71ff93cc33cf1036","afterSHA256":"621909530fe228f941698b8aa1acd5e19c591377be6a124d71ff93cc33cf1036","reason":"Same source body in authored semantic page"},{"oldId":"b-8958a274369b862dc8f9dfa4","targetId":"b-8958a274369b862dc8f9dfa4","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"d3f6a0e9d3a04e431b7a9c6faca644a0357974e570ae3eea66067da85832db98","afterSHA256":"d3f6a0e9d3a04e431b7a9c6faca644a0357974e570ae3eea66067da85832db98","reason":"Same source body in authored semantic page"},{"oldId":"b-7bb672cfdeb43ee048d663e4","targetId":"b-7bb672cfdeb43ee048d663e4","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"132ec47bbd37bd8b6573626070e54b3184a5c5a21294a10bd9ab4c419181acc7","afterSHA256":"132ec47bbd37bd8b6573626070e54b3184a5c5a21294a10bd9ab4c419181acc7","reason":"Same source body in authored semantic page"},{"oldId":"b-413a46732afbb354056a33f1","targetId":"b-413a46732afbb354056a33f1","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"9e28174c8843e3c00738239016bd81d74ee8e3b5857fa1ff02121cc770476f86","afterSHA256":"9e28174c8843e3c00738239016bd81d74ee8e3b5857fa1ff02121cc770476f86","reason":"Same source body in authored semantic page"},{"oldId":"b-125e45504f2ae164e7325a8e","targetId":"b-125e45504f2ae164e7325a8e","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"18fc193179c81a0c886864470e3271551a6d8a3537b9fc9d935e5490e732aadb","afterSHA256":"18fc193179c81a0c886864470e3271551a6d8a3537b9fc9d935e5490e732aadb","reason":"Same source body in authored semantic page"},{"oldId":"b-50874eb3edb170f20d46ffe0","targetId":"b-50874eb3edb170f20d46ffe0","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"70da78e6d74f17d276ba01f6377f5f483608d6e486db66f7ffab4af473c8c68d","afterSHA256":"70da78e6d74f17d276ba01f6377f5f483608d6e486db66f7ffab4af473c8c68d","reason":"Same source body in authored semantic page"},{"oldId":"b-0dded87c9b03cd6e5dec545b","targetId":"b-0dded87c9b03cd6e5dec545b","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"6bca41597071d63a25f16e927be8d8382b5da7c125bd9f583e5399e0520cc939","afterSHA256":"6bca41597071d63a25f16e927be8d8382b5da7c125bd9f583e5399e0520cc939","reason":"Same source body in authored semantic page"},{"oldId":"b-23bdd288e3e25bc95a32c817","targetId":"b-23bdd288e3e25bc95a32c817","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"rewrite","beforeSHA256":"a77ee9b5843b0305eb36f4a9cad8be08c2b7708ae228d53535a9946b02f8b488","afterSHA256":"76596d43d047ab4ae6858c71f16377d45d925bd1b3cb8a695d4f1606a5d9cbc1","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-eb4e2e04a1dfc7921c474d08","targetId":"b-eb4e2e04a1dfc7921c474d08","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"e85602c65ca5ff78574d827dfabe48bbd83b4b0913f2f7b07f928129a1cd5593","afterSHA256":"5fad46273be0df138b5d8cd83825e637a26e4d2054efd42ac82536606c85cd42","reason":"Same source body in authored semantic page"},{"oldId":"b-b91012e7023c0bd22065e268","targetId":"b-b91012e7023c0bd22065e268","page":"reading-block-dd1046d546e24b26aecee5957553a6e2","operation":"move","beforeSHA256":"54a440266f441a2ca24062791492c56d30a26070e099032b39ed5da18c6879ea","afterSHA256":"54a440266f441a2ca24062791492c56d30a26070e099032b39ed5da18c6879ea","reason":"Same source body in authored semantic page"},{"oldId":"b-376e2dd591f7cf494c89378d","targetId":"b-376e2dd591f7cf494c89378d","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"7cd1087c34e727cf26f3c9f759ef1904c52a96c60228e8922bcb000cfa7dc8d8","afterSHA256":"7cd1087c34e727cf26f3c9f759ef1904c52a96c60228e8922bcb000cfa7dc8d8","reason":"Same source body in authored semantic page"},{"oldId":"b-75c2350ffcad1daa90b35576","targetId":"b-75c2350ffcad1daa90b35576","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"efb06658a2c54105750282521a6f6c76787166ee696d375661365ac389ae0fa7","afterSHA256":"efb06658a2c54105750282521a6f6c76787166ee696d375661365ac389ae0fa7","reason":"Same source body in authored semantic page"},{"oldId":"b-4f5ad78284cbf34a885cf73e","targetId":"b-4f5ad78284cbf34a885cf73e","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"831626528161a3c074e9863fbc320f83c4427099d962c088e152ac90ac1b36db","afterSHA256":"831626528161a3c074e9863fbc320f83c4427099d962c088e152ac90ac1b36db","reason":"Same source body in authored semantic page"},{"oldId":"b-7b3dd6ab73edfa80401076b3","targetId":"b-7b3dd6ab73edfa80401076b3","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"7b74cc3e2f3205469e80f80e2972062d26a1c35de2e0f355d3b31ebd4564d1ad","afterSHA256":"7b74cc3e2f3205469e80f80e2972062d26a1c35de2e0f355d3b31ebd4564d1ad","reason":"Same source body in authored semantic page"},{"oldId":"b-0edcb4fa8c0005bbf810d553","targetId":"b-0edcb4fa8c0005bbf810d553","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"rewrite","beforeSHA256":"b2f1ae3ddd998d2fd4d04045f73e8ac1a983afde542afa752a36e3866cda4682","afterSHA256":"c029a7f961a8f3f4fc6629058322f9734e26c4f0f26ae8f34ba8cb501a091ec6","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-829cfa3feb2607d798d3d287","targetId":"b-829cfa3feb2607d798d3d287","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"dd655aba933eeca3efa133773efe24b93715ba055621e0986f10caa85cd381e9","afterSHA256":"08d0aa022e98bb75079211b073bb26e8245e159a9c4dcb17e3839b2d504f1d23","reason":"Same source body in authored semantic page"},{"oldId":"b-38144f2f16f0ccadcc348df7","targetId":"b-38144f2f16f0ccadcc348df7","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"707e7bc48c07443cf12149644f705a6c7c56301b0b49bf4524b81dbab3c92e29","afterSHA256":"707e7bc48c07443cf12149644f705a6c7c56301b0b49bf4524b81dbab3c92e29","reason":"Same source body in authored semantic page"},{"oldId":"b-38e5ed65beafcaaadb30c185","targetId":"b-38e5ed65beafcaaadb30c185","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"3db3903d93095f36602506f7c9a92536b7083ddc36b16b3535a3e4f55a1cdff6","afterSHA256":"3db3903d93095f36602506f7c9a92536b7083ddc36b16b3535a3e4f55a1cdff6","reason":"Same source body in authored semantic page"},{"oldId":"b-9e688d01222e5a0e30405f00","targetId":"b-9e688d01222e5a0e30405f00","page":"reading-block-1b53d32e9e684591be91385c30d46982","operation":"move","beforeSHA256":"3df86520780b10397874c31809ad84a890727b5848b6a2551cd77e3f4ffd45bd","afterSHA256":"3df86520780b10397874c31809ad84a890727b5848b6a2551cd77e3f4ffd45bd","reason":"Same source body in authored semantic page"},{"oldId":"b-b05fd56bb162db3cd270ed6e","targetId":"b-b05fd56bb162db3cd270ed6e","page":"reading-block-0bf24433979946a493db9110cc38c355","operation":"move","beforeSHA256":"998e4aa89e86386109534916c3365369ad33129026c1bad72b87fe9e4604612a","afterSHA256":"998e4aa89e86386109534916c3365369ad33129026c1bad72b87fe9e4604612a","reason":"Same source body in authored semantic page"},{"oldId":"b-9ca9d01b27f85f7c51fcad20","targetId":"b-9ca9d01b27f85f7c51fcad20","page":"reading-block-0bf24433979946a493db9110cc38c355","operation":"move","beforeSHA256":"e6f220aac2e44a39989ddf391bb915ed36cca0070722c1f760af3134ffc4f3d1","afterSHA256":"e6f220aac2e44a39989ddf391bb915ed36cca0070722c1f760af3134ffc4f3d1","reason":"Same source body in authored semantic page"},{"oldId":"b-71c3be866c43b69f01fd0202","targetId":"b-71c3be866c43b69f01fd0202","page":"reading-block-0bf24433979946a493db9110cc38c355","operation":"move","beforeSHA256":"bebca56938c02b0a6d803f51787a60adaa7a3695479f8eee343d1949bc01e733","afterSHA256":"bebca56938c02b0a6d803f51787a60adaa7a3695479f8eee343d1949bc01e733","reason":"Same source body in authored semantic page"},{"oldId":"block-dd1046d546e24b26aecee5957553a6e2--图像前缀目标三个位置分开看","targetId":"b-b55d2fb1392859a5bf2b3eea","operation":"retain-source-identity"},{"oldId":"block-dd1046d546e24b26aecee5957553a6e2--训练与推理的分界发生在前缀","targetId":"b-7bb672cfdeb43ee048d663e4","operation":"retain-source-identity"},{"oldId":"block-dd1046d546e24b26aecee5957553a6e2--已有实验确实完成了图像条件生成","targetId":"b-50874eb3edb170f20d46ffe0","operation":"retain-source-identity"},{"oldId":"block-1b53d32e9e684591be91385c30d46982--vqa问题控制读什么图像控制答什么","targetId":"b-376e2dd591f7cf494c89378d","operation":"retain-source-identity"},{"oldId":"block-0bf24433979946a493db9110cc38c355--看清图像条件再打开解码器","targetId":"b-b05fd56bb162db3cd270ed6e","operation":"retain-source-identity"}],"blocks":[{"id":"b-f04e0f6f56c9ce790f4c9dae","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-f04e0f6f56c9ce790f4c9dae","content":"# CLS 09 · 图像条件输出：描述与问答","sha256":"0869a21a5e70ead026f87cc0ba7340e1480c2d98d71496705a4271b274042716","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-opening","title":"CLS 09 · 图像条件输出：描述与问答","number":1},"sectionTitle":"CLS 09 · 图像条件输出：描述与问答","currentQuestion":"CLS 09 · 图像条件输出：描述与问答","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-86a8da8bed3881dcda2eff01","prerequisites":[{"id":"b-86a8da8bed3881dcda2eff01","url":"#b-86a8da8bed3881dcda2eff01","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-86a8da8bed3881dcda2eff01","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-86a8da8bed3881dcda2eff01","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-opening","title":"CLS 09 · 图像条件输出：描述与问答","number":1},"sectionTitle":"CLS 09 · 图像条件输出：描述与问答","currentQuestion":"CLS 09 · 图像条件输出：描述与问答","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f04e0f6f56c9ce790f4c9dae","nextBlock":"b-a6908b3621fbebcb832caf23","prerequisites":[{"id":"b-f04e0f6f56c9ce790f4c9dae","url":"#b-f04e0f6f56c9ce790f4c9dae","role":"section-title"}],"relatedBlocks":[]},{"id":"b-a6908b3621fbebcb832caf23","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-a6908b3621fbebcb832caf23","content":"图像条件、teacher forcing、真实生成、greedy/beam与VQA控制","sha256":"a192bf58b03c510999643a276b8d2261f6c9389ea09aebd6ad3419fe70e2a7b9","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-opening","title":"CLS 09 · 图像条件输出：描述与问答","number":1},"sectionTitle":"CLS 09 · 图像条件输出：描述与问答","currentQuestion":"CLS 09 · 图像条件输出：描述与问答","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-86a8da8bed3881dcda2eff01","nextBlock":"b-537c165a996846e1f65ebcd9","prerequisites":[{"id":"b-f04e0f6f56c9ce790f4c9dae","url":"#b-f04e0f6f56c9ce790f4c9dae","role":"section-title"},{"id":"b-86a8da8bed3881dcda2eff01","url":"#b-86a8da8bed3881dcda2eff01","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-537c165a996846e1f65ebcd9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-537c165a996846e1f65ebcd9","content":"可先阅读已有结果，再决定是否运行。重算时请核对各节记录的数据、参数、环境与日期。","sha256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-opening","title":"CLS 09 · 图像条件输出：描述与问答","number":1},"sectionTitle":"CLS 09 · 图像条件输出：描述与问答","currentQuestion":"CLS 09 · 图像条件输出：描述与问答","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a6908b3621fbebcb832caf23","nextBlock":null,"prerequisites":[{"id":"b-f04e0f6f56c9ce790f4c9dae","url":"#b-f04e0f6f56c9ce790f4c9dae","role":"section-title"},{"id":"b-86a8da8bed3881dcda2eff01","url":"#b-86a8da8bed3881dcda2eff01","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c8c22e4cb390d063c7e1f222","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-c8c22e4cb390d063c7e1f222","content":"**学习路径：** [图像怎样约束句子与回答](/course/study/caption-vqa.html)","sha256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-a013754450e2144dbeee4353","prerequisites":[],"relatedBlocks":[]},{"id":"b-a013754450e2144dbeee4353","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-a013754450e2144dbeee4353","content":"从[P51 看图描述](/course/pages/l3-r2-p51.html)与[P54 视觉问答](/course/pages/l3-r2-p54.html)进入：同一视觉编码器怎样服务“生成一句话”和“针对问题回答”两种输出？","sha256":"6b3101dc74d613fd527b60ff21d1ac65db81cca6fc49d581090c9a90654d2057","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c8c22e4cb390d063c7e1f222","nextBlock":"b-c2e7548ea1060e22a275417f","prerequisites":[{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c2e7548ea1060e22a275417f","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-c2e7548ea1060e22a275417f","content":"先读图像、文字前缀和下一词目标，再看VQA的同图换问/同问换图。按兴趣选择[LAB 13](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/3da47f3ee3554ee0a36e0bb6a2fdc61f)或[LAB 14](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/11d9f74871544a77b42024a9fe06b81a)；已有结果能先读，运行时各册独立从头执行。","sha256":"98588b7069ba7f92211a8e6cd5c8be5f8249ccf2425e964a426643a339f7a008","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a013754450e2144dbeee4353","nextBlock":"b-6f20cc81c7ddad3d737ea63d","prerequisites":[{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-6f20cc81c7ddad3d737ea63d","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-6f20cc81c7ddad3d737ea63d","content":"读完应能解释：为什么训练损失不能代替自由生成评价，为什么问题先验不能代替视觉证据。GRU/RNN 内部与 beam 搜索已移入[第五讲](https://deepnote.com/project/841ec8e7-e352-4840-b628-d3742b03d6d0/notebook/b1c8e2fd764441e8820b808fc40ec3cb?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=b1c8e2fd764441e8820b808fc40ec3cb&utm_term=get_notebook)，不是理解视觉条件的前置要求。","sha256":"46d053d1cc569605d484255e6af2c3466f7d40f4fe2b28cec955d4de4a9dd9d8","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c2e7548ea1060e22a275417f","nextBlock":"b-b55d2fb1392859a5bf2b3eea","prerequisites":[{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b55d2fb1392859a5bf2b3eea","type":"h2","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-b55d2fb1392859a5bf2b3eea","content":"## 图像、前缀、目标，三个位置分开看","sha256":"5d04e11d8c2e016f93a6fd003c5aafbcfa45431a430ae351461bb69f38e77a4d","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6f20cc81c7ddad3d737ea63d","nextBlock":"b-2a68c81d4c275a336fa14035","prerequisites":[{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2a68c81d4c275a336fa14035","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-2a68c81d4c275a336fa14035","content":"令图像为 $I$、句子为 $y_1,\\ldots,y_T$，模型学习\n$p(y_1,\\ldots,y_T\\mid I)=\\prod_t p(y_t\\mid I,y_{<t}).$\n视觉编码器把像素变成特征；文字预测器接收视觉特征和允许看到的文字；目标是下一词。课堂从“红衣人”后预测“拿着”，再从“红衣人拿着”后预测“球”。图上的目标词是人工参考，不是实测生成。","sha256":"621909530fe228f941698b8aa1acd5e19c591377be6a124d71ff93cc33cf1036","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"I","definitionContext":"图像、前缀、目标，三个位置分开看"},{"latex":"y_1,\\ldots,y_T","definitionContext":"图像、前缀、目标，三个位置分开看"},{"latex":"p(y_1,\\ldots,y_T\\mid I)=\\prod_t p(y_t\\mid I,y_{<t}).","definitionContext":"图像、前缀、目标，三个位置分开看"}],"previousBlock":"b-b55d2fb1392859a5bf2b3eea","nextBlock":"b-8958a274369b862dc8f9dfa4","prerequisites":[{"id":"b-b55d2fb1392859a5bf2b3eea","url":"#b-b55d2fb1392859a5bf2b3eea","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8958a274369b862dc8f9dfa4","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-8958a274369b862dc8f9dfa4","content":"一图可以有“两人在球场”与“红衣人拿球”等不同真实描述。模型学习条件下的表达分布，任务再决定必须包含哪些内容。[完整信息流与多参考解释](/course/blocks/caption-conditioned-token.html)。","sha256":"d3f6a0e9d3a04e431b7a9c6faca644a0357974e570ae3eea66067da85832db98","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2a68c81d4c275a336fa14035","nextBlock":"b-7bb672cfdeb43ee048d663e4","prerequisites":[{"id":"b-b55d2fb1392859a5bf2b3eea","url":"#b-b55d2fb1392859a5bf2b3eea","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7bb672cfdeb43ee048d663e4","type":"h2","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-7bb672cfdeb43ee048d663e4","content":"## 训练与推理的分界发生在前缀","sha256":"132ec47bbd37bd8b6573626070e54b3184a5c5a21294a10bd9ab4c419181acc7","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8958a274369b862dc8f9dfa4","nextBlock":"b-413a46732afbb354056a33f1","prerequisites":[{"id":"b-b55d2fb1392859a5bf2b3eea","url":"#b-b55d2fb1392859a5bf2b3eea","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-413a46732afbb354056a33f1","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-413a46732afbb354056a33f1","content":"参考 `[BOS,a,red,square,EOS]` 的输入是 `[BOS,a,red,square]`，目标是 `[a,red,square,EOS]`。PAD 仅为批量补齐，不计损失；EOS 是有效停止目标。损失按有效词求平均，不能把补齐格数也算入分母。","sha256":"9e28174c8843e3c00738239016bd81d74ee8e3b5857fa1ff02121cc770476f86","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7bb672cfdeb43ee048d663e4","nextBlock":"b-125e45504f2ae164e7325a8e","prerequisites":[{"id":"b-7bb672cfdeb43ee048d663e4","url":"#b-7bb672cfdeb43ee048d663e4","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-125e45504f2ae164e7325a8e","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-125e45504f2ae164e7325a8e","content":"训练时看到真实前缀叫 teacher forcing；推理必须接续自己的输出。先误说 blue 后，就要在 blue 条件下继续，不能改回参考 red。保留集 teacher-forced 交叉熵因此不能代替自由生成评价。[逐位置表与损失公式](/course/blocks/caption-shift-and-stop.html)。","sha256":"18fc193179c81a0c886864470e3271551a6d8a3537b9fc9d935e5490e732aadb","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-413a46732afbb354056a33f1","nextBlock":"b-50874eb3edb170f20d46ffe0","prerequisites":[{"id":"b-7bb672cfdeb43ee048d663e4","url":"#b-7bb672cfdeb43ee048d663e4","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-50874eb3edb170f20d46ffe0","type":"h2","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-50874eb3edb170f20d46ffe0","content":"## 已有实验确实完成了图像条件生成","sha256":"70da78e6d74f17d276ba01f6377f5f483608d6e486db66f7ffab4af473c8c68d","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-125e45504f2ae164e7325a8e","nextBlock":"b-0dded87c9b03cd6e5dec545b","prerequisites":[{"id":"b-7bb672cfdeb43ee048d663e4","url":"#b-7bb672cfdeb43ee048d663e4","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0dded87c9b03cd6e5dec545b","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-0dded87c9b03cd6e5dec545b","content":"[LAB 13](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/3da47f3ee3554ee0a36e0bb6a2fdc61f?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=3da47f3ee3554ee0a36e0bb6a2fdc61f&utm_term=get_notebook) 保存2026-09-22本地 CPU 基准：720张训练图、180张独立渲染保留图，18种语义组合两侧都出现。CNN产生48维视觉向量，文字模块采用小型GRU；本讲把它作为明确输入输出模块，内部留L05。推理不读取真实颜色/位置标签。","sha256":"6bca41597071d63a25f16e927be8d8382b5da7c125bd9f583e5399e0520cc939","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-50874eb3edb170f20d46ffe0","nextBlock":"b-23bdd288e3e25bc95a32c817","prerequisites":[{"id":"b-50874eb3edb170f20d46ffe0","url":"#b-50874eb3edb170f20d46ffe0","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-23bdd288e3e25bc95a32c817","type":"table","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-23bdd288e3e25bc95a32c817","content":"| 保留条件 | 整句精确匹配 |\n| --- | --- |\n| 正确图像，greedy / beam=3 | 100% / 100% |\n| 同一模型，像素置零 | 5.56% |\n| 同一模型，循环错配图像 | 0% |","sha256":"76596d43d047ab4ae6858c71f16377d45d925bd1b3cb8a695d4f1606a5d9cbc1","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["保留条件","整句精确匹配"],"rows":[["保留条件","整句精确匹配"],["正确图像，greedy / beam=3","100% / 100%"],["同一模型，像素置零","5.56%"],["同一模型，循环错配图像","0%"]]},"previousBlock":"b-0dded87c9b03cd6e5dec545b","nextBlock":"b-eb4e2e04a1dfc7921c474d08","prerequisites":[{"id":"b-50874eb3edb170f20d46ffe0","url":"#b-50874eb3edb170f20d46ffe0","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-eb4e2e04a1dfc7921c474d08","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-eb4e2e04a1dfc7921c474d08","content":"![同一前缀、不同图像的真实生成](/assets/display/9ebe76d49a449b9940b6470f.webp)","sha256":"5fad46273be0df138b5d8cd83825e637a26e4d2054efd42ac82536606c85cd42","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-23bdd288e3e25bc95a32c817","nextBlock":"b-b91012e7023c0bd22065e268","prerequisites":[{"id":"b-50874eb3edb170f20d46ffe0","url":"#b-50874eb3edb170f20d46ffe0","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b91012e7023c0bd22065e268","type":"p","scope":"block-dd1046d546e24b26aecee5957553a6e2","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-b91012e7023c0bd22065e268","content":"固定 `BOS a`，红方块左侧图生成 `a red square on left`，蓝圆右侧图生成 `a blue circle on right`。这支持当前有限任务使用正确视觉条件；不是自然照片或未见组合泛化，也没有显示 beam 优于 greedy。本轮整理未重跑训练。[数据、协议与证据细节](/course/blocks/caption-visual-controls.html)。","sha256":"54a440266f441a2ca24062791492c56d30a26070e099032b39ed5da18c6879ea","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-dd1046d546e24b26aecee5957553a6e2","title":"图像、前缀、目标，三个位置分开看","number":2},"sectionTitle":"图像、前缀、目标，三个位置分开看","currentQuestion":"图像、前缀、目标，三个位置分开看","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-eb4e2e04a1dfc7921c474d08","nextBlock":null,"prerequisites":[{"id":"b-50874eb3edb170f20d46ffe0","url":"#b-50874eb3edb170f20d46ffe0","role":"section-title"},{"id":"b-c8c22e4cb390d063c7e1f222","url":"#b-c8c22e4cb390d063c7e1f222","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-376e2dd591f7cf494c89378d","type":"h2","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-376e2dd591f7cf494c89378d","content":"## VQA：问题控制读什么，图像控制答什么","sha256":"7cd1087c34e727cf26f3c9f759ef1904c52a96c60228e8922bcb000cfa7dc8d8","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-75c2350ffcad1daa90b35576","prerequisites":[{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-75c2350ffcad1daa90b35576","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-75c2350ffcad1daa90b35576","content":"VQA建模 $p(a\\mid I,q)$。同图问数量、颜色、关系，需要不同信息；同一问题换相关图像，答案也应改变。输出可为有限答案分类，也可逐词生成，二者的答案范围和评价协议不同。","sha256":"efb06658a2c54105750282521a6f6c76787166ee696d375661365ac389ae0fa7","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"p(a\\mid I,q)","definitionContext":"VQA：问题控制读什么，图像控制答什么"}],"previousBlock":"b-376e2dd591f7cf494c89378d","nextBlock":"b-4f5ad78284cbf34a885cf73e","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"}],"relatedBlocks":[]},{"id":"b-4f5ad78284cbf34a885cf73e","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-4f5ad78284cbf34a885cf73e","content":"图A左红右蓝：问左色答红、问右色答蓝。换成图B左蓝右红，保持左色问题则应答蓝。若“视觉”分支只读取场景字典的 `left_color`，它是已知属性 oracle，不是从像素感知。[任务条件与VQA研究背景](/course/blocks/vqa-question-selects-information.html)。","sha256":"831626528161a3c074e9863fbc320f83c4427099d962c088e152ac90ac1b36db","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-75c2350ffcad1daa90b35576","nextBlock":"b-7b3dd6ab73edfa80401076b3","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7b3dd6ab73edfa80401076b3","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-7b3dd6ab73edfa80401076b3","content":"[LAB 14](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/11d9f74871544a77b42024a9fe06b81a?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=11d9f74871544a77b42024a9fe06b81a&utm_term=get_notebook) 已有真正读取像素和问题ID的CNN问答，以及单独训练的question-only。2026-09-22本地保留180张图、每图三问共540图问对，先按图划分再扩展问题，避免同图跨切分。","sha256":"7b74cc3e2f3205469e80f80e2972062d26a1c35de2e0f355d3b31ebd4564d1ad","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-4f5ad78284cbf34a885cf73e","nextBlock":"b-0edcb4fa8c0005bbf810d553","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0edcb4fa8c0005bbf810d553","type":"table","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-0edcb4fa8c0005bbf810d553","content":"| 保存条件 | 数量 | 左色 | 右形 | 全部 |\n| --- | --- | --- | --- | --- |\n| 图像＋问题 | 100% | 100% | 100% | 100% |\n| question-only | 50% | 33.33% | 50% | 44.44% |\n| 同模型，像素置零 | 50% | 33.33% | 50% | 44.44% |\n| 同模型，循环错配图像 | 83.33% | 50% | 0% | 44.44% |","sha256":"c029a7f961a8f3f4fc6629058322f9734e26c4f0f26ae8f34ba8cb501a091ec6","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["保存条件","数量","左色","右形","全部"],"rows":[["保存条件","数量","左色","右形","全部"],["图像＋问题","100%","100%","100%","100%"],["question-only","50%","33.33%","50%","44.44%"],["同模型，像素置零","50%","33.33%","50%","44.44%"],["同模型，循环错配图像","83.33%","50%","0%","44.44%"]]},"previousBlock":"b-7b3dd6ab73edfa80401076b3","nextBlock":"b-829cfa3feb2607d798d3d287","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-829cfa3feb2607d798d3d287","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-829cfa3feb2607d798d3d287","content":"![同图换问、同问换图](/assets/display/2f7b2168f5b21ee2095709fa.webp)","sha256":"08d0aa022e98bb75079211b073bb26e8245e159a9c4dcb17e3839b2d504f1d23","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0edcb4fa8c0005bbf810d553","nextBlock":"b-38144f2f16f0ccadcc348df7","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-38144f2f16f0ccadcc348df7","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-38144f2f16f0ccadcc348df7","content":"循环错配不是独立重抽，排列常保留数量却改变形状；总分相同掩盖不同错误。正确图像下的成功和干预后的下降，支持这个有限任务使用了视觉信息。问题是三个固定模板ID，不主张开放语言泛化。[逐问题指标的完整解释](/course/blocks/vqa-pixel-evidence.html)。","sha256":"707e7bc48c07443cf12149644f705a6c7c56301b0b49bf4524b81dbab3c92e29","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-829cfa3feb2607d798d3d287","nextBlock":"b-38e5ed65beafcaaadb30c185","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-38e5ed65beafcaaadb30c185","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-38e5ed65beafcaaadb30c185","content":"**备课模拟对话。** 学生：“无图也有44%，图像是不是不重要？”教师：“二选一、三选一、二选一的常量率平均就是4/9；同任务视觉模型100%且错配后下降，才说明新增条件有作用。”","sha256":"3db3903d93095f36602506f7c9a92536b7083ddc36b16b3535a3e4f55a1cdff6","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-38144f2f16f0ccadcc348df7","nextBlock":"b-9e688d01222e5a0e30405f00","prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9e688d01222e5a0e30405f00","type":"p","scope":"block-1b53d32e9e684591be91385c30d46982","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-9e688d01222e5a0e30405f00","content":"本轮没有重跑这些实验。Caption产生新句子，匹配返回候选分数，VQA还要根据问题选择信息。理解区别后，按需要进入CLS 10 的匹配或 CLS 11 的区域指代，不必把所有册重新读一遍。","sha256":"3df86520780b10397874c31809ad84a890727b5848b6a2551cd77e3f4ffd45bd","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-1b53d32e9e684591be91385c30d46982","title":"VQA：问题控制读什么，图像控制答什么","number":3},"sectionTitle":"VQA：问题控制读什么，图像控制答什么","currentQuestion":"VQA：问题控制读什么，图像控制答什么","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-38e5ed65beafcaaadb30c185","nextBlock":null,"prerequisites":[{"id":"b-376e2dd591f7cf494c89378d","url":"#b-376e2dd591f7cf494c89378d","role":"section-title"},{"id":"b-75c2350ffcad1daa90b35576","url":"#b-75c2350ffcad1daa90b35576","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b05fd56bb162db3cd270ed6e","type":"h2","scope":"block-0bf24433979946a493db9110cc38c355","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-b05fd56bb162db3cd270ed6e","content":"## 看清图像条件，再打开解码器","sha256":"998e4aa89e86386109534916c3365369ad33129026c1bad72b87fe9e4604612a","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-0bf24433979946a493db9110cc38c355","title":"看清图像条件，再打开解码器","number":4},"sectionTitle":"看清图像条件，再打开解码器","currentQuestion":"看清图像条件，再打开解码器","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-9ca9d01b27f85f7c51fcad20","prerequisites":[{"id":"b-9ca9d01b27f85f7c51fcad20","url":"#b-9ca9d01b27f85f7c51fcad20","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9ca9d01b27f85f7c51fcad20","type":"p","scope":"block-0bf24433979946a493db9110cc38c355","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-9ca9d01b27f85f7c51fcad20","content":"本讲先检查图像是否真正影响逐词预测与生成结果：[LAB 13](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/3da47f3ee3554ee0a36e0bb6a2fdc61f?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=3da47f3ee3554ee0a36e0bb6a2fdc61f&utm_term=get_notebook)保留训练、实际接续输出、遮图与错图消融。较高模型概率和更符合图片是两个评价问题，不能由搜索方式直接互相保证。","sha256":"e6f220aac2e44a39989ddf391bb915ed36cca0070722c1f760af3134ffc4f3d1","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-0bf24433979946a493db9110cc38c355","title":"看清图像条件，再打开解码器","number":4},"sectionTitle":"看清图像条件，再打开解码器","currentQuestion":"看清图像条件，再打开解码器","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b05fd56bb162db3cd270ed6e","nextBlock":"b-71c3be866c43b69f01fd0202","prerequisites":[{"id":"b-b05fd56bb162db3cd270ed6e","url":"#b-b05fd56bb162db3cd270ed6e","role":"section-title"}],"relatedBlocks":[]},{"id":"b-71c3be866c43b69f01fd0202","type":"p","scope":"block-0bf24433979946a493db9110cc38c355","url":"https://codingai-lec03.pages.dev/course/cls10-图像描述解码与视觉问答.html#b-71c3be866c43b69f01fd0202","content":"循环状态方程、整句概率分解、规范化概率树及 greedy/beam 比较，已整理到[第五讲：序列模型与图像条件解码](https://deepnote.com/project/841ec8e7-e352-4840-b628-d3742b03d6d0/notebook/b1c8e2fd764441e8820b808fc40ec3cb?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=b1c8e2fd764441e8820b808fc40ec3cb&utm_term=get_notebook)。需要解释 LAB 13 的内部实现时再沿此入口展开。","sha256":"bebca56938c02b0a6d803f51787a60adaa7a3695479f8eee343d1949bc01e733","chapter":{"id":"course","title":"CLS 09 · 图像条件输出：描述与问答"},"unit":{"id":"cls10-图像描述解码与视觉问答","title":"CLS 09 · 图像条件输出：描述与问答"},"page":{"id":"reading-block-0bf24433979946a493db9110cc38c355","title":"看清图像条件，再打开解码器","number":4},"sectionTitle":"看清图像条件，再打开解码器","currentQuestion":"看清图像条件，再打开解码器","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9ca9d01b27f85f7c51fcad20","nextBlock":null,"prerequisites":[{"id":"b-b05fd56bb162db3cd270ed6e","url":"#b-b05fd56bb162db3cd270ed6e","role":"section-title"},{"id":"b-9ca9d01b27f85f7c51fcad20","url":"#b-9ca9d01b27f85f7c51fcad20","role":"problem-context"}],"relatedBlocks":[]}]}