{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"LAB 14 · 图像问答：视觉条件与问题先验","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html","markdown":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.md","pages":[{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":1},{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":2},{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":3},{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":4},{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":5},{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":6},{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":7},{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","unit":"语言怎样选择当前需要的视觉信息？","unitId":"language-local-and-question","chapter":"III","number":8}],"aliases":[{"oldId":"b-80c8841fefb3e0adeac1c102","targetId":"b-80c8841fefb3e0adeac1c102","page":"reading-opening","operation":"move","beforeSHA256":"1f37c330374145c33cdc473dedf57906452b35f19f323be695280cb12f17875b","afterSHA256":"1f37c330374145c33cdc473dedf57906452b35f19f323be695280cb12f17875b","reason":"Same source body in authored semantic page"},{"oldId":"b-93417eafac1ad416361ec0d7","targetId":"b-93417eafac1ad416361ec0d7","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-33f62ca0524e04b0d7090741","targetId":"b-33f62ca0524e04b0d7090741","page":"reading-opening","operation":"move","beforeSHA256":"5790b0afb52508f0174b1bc581f7847035a51c5c2a46e5c821bcd9d5d61bcbb0","afterSHA256":"5790b0afb52508f0174b1bc581f7847035a51c5c2a46e5c821bcd9d5d61bcbb0","reason":"Same source body in authored semantic page"},{"oldId":"b-a70e1f7ddb8db4b6186f0b08","targetId":"b-a70e1f7ddb8db4b6186f0b08","page":"reading-opening","operation":"move","beforeSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","afterSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","reason":"Same source body in authored semantic page"},{"oldId":"b-81c4998f912e190cc015af5f","targetId":"b-81c4998f912e190cc015af5f","page":"reading-opening","operation":"move","beforeSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","afterSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","reason":"Same source body in authored semantic page"},{"oldId":"b-dab46a1aa74bacaa7323d7de","targetId":"b-dab46a1aa74bacaa7323d7de","page":"reading-opening","operation":"move","beforeSHA256":"54bacb112e17b840a5a310ce968bdae21f53376a4ba54a33e651576ea0a41a11","afterSHA256":"54bacb112e17b840a5a310ce968bdae21f53376a4ba54a33e651576ea0a41a11","reason":"Same source body in authored semantic page"},{"oldId":"b-932858cfc7f7a9bd702fa990","targetId":"b-932858cfc7f7a9bd702fa990","page":"reading-opening","operation":"move","beforeSHA256":"a39bb3bfff571024e21cbf171c189671131c82900779f89b74b1609b897f31b1","afterSHA256":"a39bb3bfff571024e21cbf171c189671131c82900779f89b74b1609b897f31b1","reason":"Same source body in authored semantic page"},{"oldId":"b-ab7e7c224cbad9efcc25f4e6","targetId":"b-ab7e7c224cbad9efcc25f4e6","page":"reading-opening","operation":"move","beforeSHA256":"4790c4b243b6afc33a3b0a94b904dcef5a39431db1ef3a267d3dec7d13a4df01","afterSHA256":"4790c4b243b6afc33a3b0a94b904dcef5a39431db1ef3a267d3dec7d13a4df01","reason":"Same source body in authored semantic page"},{"oldId":"b-9a5f40919931774f91518d03","targetId":"b-9a5f40919931774f91518d03","page":"reading-opening","operation":"move","beforeSHA256":"a2c341314db4e1d785717eeb1afabfbccce760d49244c5ce15bae05cabd5117a","afterSHA256":"9a2655d1601e5f06db61c1f983135287b33ee7b26e382647125686614f100163","reason":"Same source body in authored semantic page"},{"oldId":"b-10492f36c7445d023c22eaba","targetId":"b-10492f36c7445d023c22eaba","page":"reading-opening","operation":"move","beforeSHA256":"36843086e2b941e4d4724082772339fb7dc1f03fc1e5596c7f355166ed31e988","afterSHA256":"36843086e2b941e4d4724082772339fb7dc1f03fc1e5596c7f355166ed31e988","reason":"Same source body in authored semantic page"},{"oldId":"b-7bb7cbce72f4633a081924de","targetId":"b-7bb7cbce72f4633a081924de","page":"reading-block-4919dea7730a4e19823525d3b77d89c9","operation":"move","beforeSHA256":"8561dd038e5430bf28bca9be060c043b2540da69b265d2984f130ede699934a5","afterSHA256":"8561dd038e5430bf28bca9be060c043b2540da69b265d2984f130ede699934a5","reason":"Same source body in authored semantic page"},{"oldId":"b-35a9ad5ff62e87e3b5d710c1","targetId":"b-35a9ad5ff62e87e3b5d710c1","page":"reading-block-4919dea7730a4e19823525d3b77d89c9","operation":"move","beforeSHA256":"35402536e21a2aa99b1809d10824524d03cd182909456a61a350c6b6aa2aad8f","afterSHA256":"35402536e21a2aa99b1809d10824524d03cd182909456a61a350c6b6aa2aad8f","reason":"Same source body in authored semantic page"},{"oldId":"b-af24c85e3f508fc22ce43733","targetId":"b-af24c85e3f508fc22ce43733","page":"reading-block-4919dea7730a4e19823525d3b77d89c9","operation":"move","beforeSHA256":"c87a6ce586309b464659eaf822eb0ff4d176127095e5ad36a39eed58c5928689","afterSHA256":"c87a6ce586309b464659eaf822eb0ff4d176127095e5ad36a39eed58c5928689","reason":"Same source body in authored semantic page"},{"oldId":"b-8f7a201ac3ed9b291eebea14","targetId":"b-8f7a201ac3ed9b291eebea14","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"b6acdcaa9ced1ec6dd65e6de6981dbcd628d7bfd828f0f12c62297e0630bc40c","afterSHA256":"b6acdcaa9ced1ec6dd65e6de6981dbcd628d7bfd828f0f12c62297e0630bc40c","reason":"Same source body in authored semantic page"},{"oldId":"b-f07fff48edb90c4ae3c12b08","targetId":"b-f07fff48edb90c4ae3c12b08","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"23b2824dee5d302542c4d7c618677b4934afb306b67f64f4ca576b3065f2a57f","afterSHA256":"23b2824dee5d302542c4d7c618677b4934afb306b67f64f4ca576b3065f2a57f","reason":"Same source body in authored semantic page"},{"oldId":"b-da37597f1a9de82e238085f6","targetId":"b-da37597f1a9de82e238085f6","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"86d6ce3d14d4990ac0f81c69516fc4d095213a1a3b4633f5d8aa2e0196bc90fd","afterSHA256":"86d6ce3d14d4990ac0f81c69516fc4d095213a1a3b4633f5d8aa2e0196bc90fd","reason":"Same source body in authored semantic page"},{"oldId":"b-c68f33abf9adb345238f4c4e","targetId":"b-c68f33abf9adb345238f4c4e","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"810908fd69927093502edf9303e0f30160207a520fff2e154f31763f77c1dc30","afterSHA256":"810908fd69927093502edf9303e0f30160207a520fff2e154f31763f77c1dc30","reason":"Same source body in authored semantic page"},{"oldId":"b-8fe61110e25c104b4f431314","targetId":"b-8fe61110e25c104b4f431314","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"3755eb55c18436f73604f02646360ddd44f5e051c982148d4ea071c8165155ea","afterSHA256":"3755eb55c18436f73604f02646360ddd44f5e051c982148d4ea071c8165155ea","reason":"Same source body in authored semantic page"},{"oldId":"b-d4191c62414693f951ddde0b","targetId":"b-d4191c62414693f951ddde0b","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"70bc354d398d992dadd4827ac6b23f9133c0a8c443756ad4656e7371c41512d5","afterSHA256":"70bc354d398d992dadd4827ac6b23f9133c0a8c443756ad4656e7371c41512d5","reason":"Same source body in authored semantic page"},{"oldId":"b-9e609bf96c4e93aa06e026da","targetId":"b-9e609bf96c4e93aa06e026da","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"4e4b94606681bb6d2eb7b4afda09881a8d3ee251bb26915bd8dab24867b299a2","afterSHA256":"4e4b94606681bb6d2eb7b4afda09881a8d3ee251bb26915bd8dab24867b299a2","reason":"Same source body in authored semantic page"},{"oldId":"b-d15e6926903f560fa547d019","targetId":"b-d15e6926903f560fa547d019","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"25a797783c0d33246a29e6b05de2a88236b882aa0783b1729ebf8616970cba5d","afterSHA256":"25a797783c0d33246a29e6b05de2a88236b882aa0783b1729ebf8616970cba5d","reason":"Same source body in authored semantic page"},{"oldId":"b-a962abd04c046a7c528efcd3","targetId":"b-a962abd04c046a7c528efcd3","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"36c815acc00c365f774968586f380e1fed0fcacf2b8ed3f268abf3b2845d3299","afterSHA256":"36c815acc00c365f774968586f380e1fed0fcacf2b8ed3f268abf3b2845d3299","reason":"Same source body in authored semantic page"},{"oldId":"b-8824392af1bda1df08d061ac","targetId":"b-8824392af1bda1df08d061ac","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"3129d617c9d64f12477cc585d4a415fb35853a79d91697ee3a67e21cf439afc7","afterSHA256":"3129d617c9d64f12477cc585d4a415fb35853a79d91697ee3a67e21cf439afc7","reason":"Same source body in authored semantic page"},{"oldId":"b-6c6dcbedc229d455a09a9136","targetId":"b-6c6dcbedc229d455a09a9136","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"bd1543535de10e14989b8f47346a883f97b200294db444f7b421779ebcfdcd67","afterSHA256":"bd1543535de10e14989b8f47346a883f97b200294db444f7b421779ebcfdcd67","reason":"Same source body in authored semantic page"},{"oldId":"b-1059308191d5c884039d53ec","targetId":"b-1059308191d5c884039d53ec","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"8eef598675a5a3470afd723119d407b25beb4b74308e2af413c81ebdca7c6581","afterSHA256":"8eef598675a5a3470afd723119d407b25beb4b74308e2af413c81ebdca7c6581","reason":"Same source body in authored semantic page"},{"oldId":"b-a4cb3ff6602b195295b683a6","targetId":"b-a4cb3ff6602b195295b683a6","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"701155870a1e8d1c9076048a018c3eac278a5c5ce5d1530da54c7367d1727177","afterSHA256":"701155870a1e8d1c9076048a018c3eac278a5c5ce5d1530da54c7367d1727177","reason":"Same source body in authored semantic page"},{"oldId":"b-18be02fd14ea8f6ad6adc9d2","targetId":"b-18be02fd14ea8f6ad6adc9d2","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"1fe68838bac1f3e94024769f31316795734a73b8163daa4afe9dffeb19a16c42","afterSHA256":"1fe68838bac1f3e94024769f31316795734a73b8163daa4afe9dffeb19a16c42","reason":"Same source body in authored semantic page"},{"oldId":"b-5a4380a2ba607f596eb8bdd9","targetId":"b-5a4380a2ba607f596eb8bdd9","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"rewrite","beforeSHA256":"1a66cf7d55a82b0f7527859ddd0e5368c56f264de60399a8230e2de00d0acaba","afterSHA256":"c973352dfecd3c1e7c8a26ed089a8951087a83f682f227b3de93c8dcc5168934","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-5e7abc5bad9c9d07627a1294","targetId":"b-5e7abc5bad9c9d07627a1294","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"5772517b1201f9efe13bb04cb40965793bbfb2044defb920a18a4690ca93b8af","afterSHA256":"5772517b1201f9efe13bb04cb40965793bbfb2044defb920a18a4690ca93b8af","reason":"Same source body in authored semantic page"},{"oldId":"b-ffec4b0439146f71b12329c6","targetId":"b-ffec4b0439146f71b12329c6","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"af9db8f48afeeba125b9490230557bd0aa98c406e67197f8697291fd1cf52a14","afterSHA256":"96dafa969bed5d9ad470f33155f80bcacac14d744136bb6b734389f81e9e5cf9","reason":"Same source body in authored semantic page"},{"oldId":"b-3b7226e13debda82f9b6a776","targetId":"b-3b7226e13debda82f9b6a776","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"c51336f20e1d778413d93dfee9689f8717a4228ddc7f0a4593271786481d265b","afterSHA256":"c51336f20e1d778413d93dfee9689f8717a4228ddc7f0a4593271786481d265b","reason":"Same source body in authored semantic page"},{"oldId":"b-fb0a6b5386c1ffc96594c11b","targetId":"b-fb0a6b5386c1ffc96594c11b","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"8e18a29d7fa6b2a327c610746be654c9657a1db37fbbcdf87d48868eec182c2d","afterSHA256":"8e18a29d7fa6b2a327c610746be654c9657a1db37fbbcdf87d48868eec182c2d","reason":"Same source body in authored semantic page"},{"oldId":"b-a9f9b9ab7bb4a4f29b95d3e2","targetId":"b-a9f9b9ab7bb4a4f29b95d3e2","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"1ca0d2332e22d968ff90ecec13d4503cd081f51138359dfa0245b7402b49375f","afterSHA256":"1ca0d2332e22d968ff90ecec13d4503cd081f51138359dfa0245b7402b49375f","reason":"Same source body in authored semantic page"},{"oldId":"b-2954f6eb6b4876cee8d0a83e","targetId":"b-2954f6eb6b4876cee8d0a83e","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"27958004cbb247516adb951070045ae59fb33da0aa36cc1ec01cdab2c166f0e0","afterSHA256":"ef326977eb3c2d61ef5d4269aeb7b71d502b9f00abd948c34e3a83fd1eff4bba","reason":"Same source body in authored semantic page"},{"oldId":"b-736d22f1f6ed4a061495a2c3","targetId":"b-736d22f1f6ed4a061495a2c3","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"beea7d2078e665fc442333a2e15bb69694a025ec8f0b7eb6e2db5b2861eb3a42","afterSHA256":"beea7d2078e665fc442333a2e15bb69694a025ec8f0b7eb6e2db5b2861eb3a42","reason":"Same source body in authored semantic page"},{"oldId":"b-f791f41befe36614964c608e","targetId":"b-f791f41befe36614964c608e","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"0632888c3952b95db1c950e8a9167e33450ef30280bad500214018db5eedcf4b","afterSHA256":"0632888c3952b95db1c950e8a9167e33450ef30280bad500214018db5eedcf4b","reason":"Same source body in authored semantic page"},{"oldId":"b-8e42f91101cbaab36fcc84cd","targetId":"b-8e42f91101cbaab36fcc84cd","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"2e78f928663ed19a4abe9bb865f2ec5b12eca912e29834ec3865f05c50ab3b0b","afterSHA256":"2e78f928663ed19a4abe9bb865f2ec5b12eca912e29834ec3865f05c50ab3b0b","reason":"Same source body in authored semantic page"},{"oldId":"b-701e3fb20f18c27420592e60","targetId":"b-701e3fb20f18c27420592e60","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"8660b9343b7f1d52be6bcc92613183fa8e5476cd47e6bec2381d18175cddfcca","afterSHA256":"8660b9343b7f1d52be6bcc92613183fa8e5476cd47e6bec2381d18175cddfcca","reason":"Same source body in authored semantic page"},{"oldId":"b-0931a8ad2f714e26836deb30","targetId":"b-0931a8ad2f714e26836deb30","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"be9dcfab95622299ac41f14246ffc693f6f04b7a84339f461ae3821b17c802ec","afterSHA256":"be9dcfab95622299ac41f14246ffc693f6f04b7a84339f461ae3821b17c802ec","reason":"Same source body in authored semantic page"},{"oldId":"b-ecb0f2afeb1daf83541dd0a4","targetId":"b-ecb0f2afeb1daf83541dd0a4","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"605e525420770ca13650761ad0f9b0d9e751f8f5a21f8e29f246e4562f2f4a9e","afterSHA256":"605e525420770ca13650761ad0f9b0d9e751f8f5a21f8e29f246e4562f2f4a9e","reason":"Same source body in authored semantic page"},{"oldId":"b-e552553b9a0a852b47ca618f","targetId":"b-e552553b9a0a852b47ca618f","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"4bdd15d61276026eb047e031a5db6e0226050b63c5c87eeb2a31ae52cb226668","afterSHA256":"4bdd15d61276026eb047e031a5db6e0226050b63c5c87eeb2a31ae52cb226668","reason":"Same source body in authored semantic page"},{"oldId":"block-4919dea7730a4e19823525d3b77d89c9--0-先给原例子正确命名","targetId":"b-7bb7cbce72f4633a081924de","operation":"retain-source-identity"},{"oldId":"block-3c10b348d7d1412b80d07fc5583e1a27--1-环境与有限问题接口","targetId":"b-8f7a201ac3ed9b291eebea14","operation":"retain-source-identity"},{"oldId":"block-20b57075949948ac8d6711cd4621f5b6--2-先按图像切分避免同一图的不同问题泄漏","targetId":"b-8fe61110e25c104b4f431314","operation":"retain-source-identity"},{"oldId":"block-62876c0daff44e6f9b582bcaa0d0dd00--3-模型损失与公平比较范围","targetId":"b-8824392af1bda1df08d061ac","operation":"retain-source-identity"},{"oldId":"block-4eb1163b8bc64dd5ab088d22e10318bb--4-逐问题读结果别只看总准确率","targetId":"b-18be02fd14ea8f6ad6adc9d2","operation":"retain-source-identity"},{"oldId":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8--5-学习曲线与可以被推翻的结论","targetId":"b-a9f9b9ab7bb4a4f29b95d3e2","operation":"retain-source-identity"},{"oldId":"block-193cc04bc38242b7be21838002bf9285--结果文件来源与回链","targetId":"b-701e3fb20f18c27420592e60","operation":"retain-source-identity"}],"blocks":[{"id":"b-80c8841fefb3e0adeac1c102","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-80c8841fefb3e0adeac1c102","content":"# LAB 14 · 图像问答：视觉条件与问题先验","sha256":"1f37c330374145c33cdc473dedf57906452b35f19f323be695280cb12f17875b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-93417eafac1ad416361ec0d7","prerequisites":[{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-93417eafac1ad416361ec0d7","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-93417eafac1ad416361ec0d7","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-80c8841fefb3e0adeac1c102","nextBlock":"b-33f62ca0524e04b0d7090741","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"}],"relatedBlocks":[]},{"id":"b-33f62ca0524e04b0d7090741","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-33f62ca0524e04b0d7090741","content":"RGB+问题输入、question-only对照、同图换问与同问换图","sha256":"5790b0afb52508f0174b1bc581f7847035a51c5c2a46e5c821bcd9d5d61bcbb0","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-93417eafac1ad416361ec0d7","nextBlock":"b-a70e1f7ddb8db4b6186f0b08","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a70e1f7ddb8db4b6186f0b08","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-a70e1f7ddb8db4b6186f0b08","content":"可先阅读已有结果，再决定是否运行。重算时请核对各节记录的数据、参数、环境与日期。","sha256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-33f62ca0524e04b0d7090741","nextBlock":"b-81c4998f912e190cc015af5f","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-81c4998f912e190cc015af5f","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-81c4998f912e190cc015af5f","content":"**学习路径：** [图像怎样约束句子与回答](/course/study/caption-vqa.html)","sha256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a70e1f7ddb8db4b6186f0b08","nextBlock":"b-dab46a1aa74bacaa7323d7de","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-dab46a1aa74bacaa7323d7de","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-dab46a1aa74bacaa7323d7de","content":"同图换问题，模型读什么会变；同问换图片，答案也应跟着变。先读[P54](/course/pages/l3-r2-p54.html)，再看第2节按图切分和第4节逐问题结果。","sha256":"54bacb112e17b840a5a310ce968bdae21f53376a4ba54a33e651576ea0a41a11","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-81c4998f912e190cc015af5f","nextBlock":"b-932858cfc7f7a9bd702fa990","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-932858cfc7f7a9bd702fa990","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-932858cfc7f7a9bd702fa990","content":"开头字典例是已知属性oracle，主体才是读取RGB像素与问题ID的CNN，并单独训练question-only。三类问题为数量、左色、右形，答案为有限词表；这不等于开放语言问答。[完整控制和证据](/course/blocks/vqa-pixel-evidence.html)。","sha256":"a39bb3bfff571024e21cbf171c189671131c82900779f89b74b1609b897f31b1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-dab46a1aa74bacaa7323d7de","nextBlock":"b-ab7e7c224cbad9efcc25f4e6","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ab7e7c224cbad9efcc25f4e6","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-ab7e7c224cbad9efcc25f4e6","content":"读完应能解释question-only为何约44.44%，并从同图换问/同问换图各拿一个例子支持“模型使用了条件”。只读可先看2026-09-22保存结果，运行则按全部代码顺序；本次整理不改历史成绩。若答案需要一个区域而非类别，继续[LAB 15](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/7dc83786d067465aa76264475db2c62d)。","sha256":"4790c4b243b6afc33a3b0a94b904dcef5a39431db1ef3a267d3dec7d13a4df01","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-932858cfc7f7a9bd702fa990","nextBlock":"b-9a5f40919931774f91518d03","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9a5f40919931774f91518d03","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-9a5f40919931774f91518d03","content":"![同图换问与同问换图](/assets/display/2f7b2168f5b21ee2095709fa.webp)","sha256":"9a2655d1601e5f06db61c1f983135287b33ee7b26e382647125686614f100163","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ab7e7c224cbad9efcc25f4e6","nextBlock":"b-10492f36c7445d023c22eaba","prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-10492f36c7445d023c22eaba","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-10492f36c7445d023c22eaba","content":"前两幅共享形状、位置和随机噪声，只改左侧颜色；第三幅只用于展示另一个问题条件，不是与前两幅只改一项。想让答案直接成为一个区域，继续 [LAB 15 · 语言指代小练习：目标、参照与关系](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/7dc83786d067465aa76264475db2c62d?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=7dc83786d067465aa76264475db2c62d&utm_term=get_notebook)。","sha256":"36843086e2b941e4d4724082772339fb7dc1f03fc1e5596c7f355166ed31e988","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9a5f40919931774f91518d03","nextBlock":null,"prerequisites":[{"id":"b-80c8841fefb3e0adeac1c102","url":"#b-80c8841fefb3e0adeac1c102","role":"section-title"},{"id":"b-93417eafac1ad416361ec0d7","url":"#b-93417eafac1ad416361ec0d7","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7bb7cbce72f4633a081924de","type":"h2","scope":"block-4919dea7730a4e19823525d3b77d89c9","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-7bb7cbce72f4633a081924de","content":"## 0. 先给原例子正确命名","sha256":"8561dd038e5430bf28bca9be060c043b2540da69b265d2984f130ede699934a5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","number":2},"sectionTitle":"0. 先给原例子正确命名","currentQuestion":"0. 先给原例子正确命名","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-35a9ad5ff62e87e3b5d710c1","prerequisites":[{"id":"b-35a9ad5ff62e87e3b5d710c1","url":"#b-35a9ad5ff62e87e3b5d710c1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-35a9ad5ff62e87e3b5d710c1","type":"p","scope":"block-4919dea7730a4e19823525d3b77d89c9","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-35a9ad5ff62e87e3b5d710c1","content":"原 `with-image` 分支读取字典中的 `cups` 与 `left`。这些已是答案属性，因此此处保留为 **known-attribute oracle**。它能说明问题选择不同字段，却没有完成视觉感知。下面打印原例，再与真正从像素学习的模型区分。","sha256":"35402536e21a2aa99b1809d10824524d03cd182909456a61a350c6b6aa2aad8f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","number":2},"sectionTitle":"0. 先给原例子正确命名","currentQuestion":"0. 先给原例子正确命名","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7bb7cbce72f4633a081924de","nextBlock":"b-af24c85e3f508fc22ce43733","prerequisites":[{"id":"b-7bb7cbce72f4633a081924de","url":"#b-7bb7cbce72f4633a081924de","role":"section-title"}],"relatedBlocks":[]},{"id":"b-af24c85e3f508fc22ce43733","type":"pre","scope":"block-ee681c33c81c4de98ecea27d981ddb73","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-af24c85e3f508fc22ce43733","content":"```\n# Detached readonly runs write only to an isolated ephemeral directory.\nimport os, tempfile\n_LAB_WORKDIR = tempfile.mkdtemp(prefix=\"lecture03-lab14-\")\nos.chdir(_LAB_WORKDIR)\nprint(\"ephemeral workdir:\", _LAB_WORKDIR)\n\ncases=[{'cups':2,'left':'red','q':'count'},{'cups':2,'left':'red','q':'left-color'},{'cups':2,'left':'blue','q':'left-color'}]\nfor c in cases:\n    prior='2' if c['q']=='count' else 'red'\n    oracle=str(c['cups']) if c['q']=='count' else c['left']\n    print(c,'question-only rule',prior,'known-attribute oracle',oracle)\n# No image was read above. The learned visual path begins below.\n```","sha256":"c87a6ce586309b464659eaf822eb0ff4d176127095e5ad36a39eed58c5928689","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","number":2},"sectionTitle":"0. 先给原例子正确命名","currentQuestion":"0. 先给原例子正确命名","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-35a9ad5ff62e87e3b5d710c1","nextBlock":null,"prerequisites":[{"id":"b-7bb7cbce72f4633a081924de","url":"#b-7bb7cbce72f4633a081924de","role":"section-title"},{"id":"b-35a9ad5ff62e87e3b5d710c1","url":"#b-35a9ad5ff62e87e3b5d710c1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8f7a201ac3ed9b291eebea14","type":"h2","scope":"block-3c10b348d7d1412b80d07fc5583e1a27","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-8f7a201ac3ed9b291eebea14","content":"## 1. 环境与有限问题接口","sha256":"b6acdcaa9ced1ec6dd65e6de6981dbcd628d7bfd828f0f12c62297e0630bc40c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-f07fff48edb90c4ae3c12b08","prerequisites":[{"id":"b-f07fff48edb90c4ae3c12b08","url":"#b-f07fff48edb90c4ae3c12b08","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-f07fff48edb90c4ae3c12b08","type":"p","scope":"block-3c10b348d7d1412b80d07fc5583e1a27","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-f07fff48edb90c4ae3c12b08","content":"依赖 NumPy、Pillow、Matplotlib、PyTorch；Restart & Run All 后无需别册内存。输出保存在 `outputs/lab14/`。2026-09-22 本地 CPU 已执行，PyTorch 2.14.0+cpu，约十余秒；不代表 Deepnote 云端执行。","sha256":"23b2824dee5d302542c4d7c618677b4934afb306b67f64f4ca576b3065f2a57f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8f7a201ac3ed9b291eebea14","nextBlock":"b-da37597f1a9de82e238085f6","prerequisites":[{"id":"b-8f7a201ac3ed9b291eebea14","url":"#b-8f7a201ac3ed9b291eebea14","role":"section-title"}],"relatedBlocks":[]},{"id":"b-da37597f1a9de82e238085f6","type":"p","scope":"block-3c10b348d7d1412b80d07fc5583e1a27","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-da37597f1a9de82e238085f6","content":"我们定义三种英文模板，用 ID 0/1/2 输入嵌入层。文字是模板的可读形式，模型不支持任意自然语言改写。答案空间为 one、two、red、blue、green、square、circle。这是封闭词表 VQA，足以检验条件依赖，但不能证明开放问答、常识推理或语言理解。","sha256":"86d6ce3d14d4990ac0f81c69516fc4d095213a1a3b4633f5d8aa2e0196bc90fd","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f07fff48edb90c4ae3c12b08","nextBlock":"b-c68f33abf9adb345238f4c4e","prerequisites":[{"id":"b-8f7a201ac3ed9b291eebea14","url":"#b-8f7a201ac3ed9b291eebea14","role":"section-title"},{"id":"b-f07fff48edb90c4ae3c12b08","url":"#b-f07fff48edb90c4ae3c12b08","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c68f33abf9adb345238f4c4e","type":"pre","scope":"block-924df89109464fed8b7817f3ebf2b233","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-c68f33abf9adb345238f4c4e","content":"```\nfrom pathlib import Path\nimport json, hashlib, platform, sys, time\nfrom datetime import datetime, timezone\nimport numpy as np\nfrom PIL import Image, ImageDraw\nimport matplotlib\nif 'get_ipython' not in globals(): matplotlib.use('Agg')\nimport matplotlib.pyplot as plt\nimport torch\nfrom torch import nn\nimport torch.nn.functional as F\nOUT=Path('outputs/lab14');OUT.mkdir(parents=True,exist_ok=True)\ntorch.set_num_threads(1);torch.manual_seed(1403);np.random.seed(1403);START=time.perf_counter()\nCOLORS={'red':(220,55,55),'blue':(45,95,225),'green':(35,170,75)}\nQUESTIONS=['how many objects?','what color is the leftmost object?','what shape is the rightmost object?']\nANSWERS=['one','two','red','blue','green','square','circle']; A={x:i for i,x in enumerate(ANSWERS)}\nprint('CPU',torch.__version__,'seed 1403; answer vocabulary:',ANSWERS)\n```","sha256":"810908fd69927093502edf9303e0f30160207a520fff2e154f31763f77c1dc30","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-da37597f1a9de82e238085f6","nextBlock":null,"prerequisites":[{"id":"b-8f7a201ac3ed9b291eebea14","url":"#b-8f7a201ac3ed9b291eebea14","role":"section-title"},{"id":"b-f07fff48edb90c4ae3c12b08","url":"#b-f07fff48edb90c4ae3c12b08","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8fe61110e25c104b4f431314","type":"h2","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-8fe61110e25c104b4f431314","content":"## 2. 先按图像切分，避免同一图的不同问题泄漏","sha256":"3755eb55c18436f73604f02646360ddd44f5e051c982148d4ea071c8165155ea","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-d4191c62414693f951ddde0b","prerequisites":[{"id":"b-d4191c62414693f951ddde0b","url":"#b-d4191c62414693f951ddde0b","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d4191c62414693f951ddde0b","type":"p","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-d4191c62414693f951ddde0b","content":"每张图含一个或两个带颜色的方形/圆形。所有图都提出同样三个问题：数量、最左物体颜色、最右物体形状；只有一个物体时，它同时是最左与最右。目标由生成器标注，这是监督数据的来源，不是推理接口。`forward(pixels,question_id)` 无法读取场景字典。","sha256":"70bc354d398d992dadd4827ac6b23f9133c0a8c443756ad4656e7371c41512d5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8fe61110e25c104b4f431314","nextBlock":"b-9e609bf96c4e93aa06e026da","prerequisites":[{"id":"b-8fe61110e25c104b4f431314","url":"#b-8fe61110e25c104b4f431314","role":"section-title"}],"relatedBlocks":[]},{"id":"b-9e609bf96c4e93aa06e026da","type":"p","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-9e609bf96c4e93aa06e026da","content":"训练图 seed 14031、保留图 seed 14032；先得到 720/180 张互不相同的图，再扩展为 2160/540 个问答。训练左侧颜色以 75% 概率采红，实际本次为 73.61%；保留集三色各占三分之一。数量与形状平衡设计下各有两种可能。语言模型因此能在训练颜色问题上靠红色先验得高分，却没有随图改变的证据。","sha256":"4e4b94606681bb6d2eb7b4afda09881a8d3ee251bb26915bd8dab24867b299a2","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d4191c62414693f951ddde0b","nextBlock":"b-d15e6926903f560fa547d019","prerequisites":[{"id":"b-8fe61110e25c104b4f431314","url":"#b-8fe61110e25c104b4f431314","role":"section-title"},{"id":"b-d4191c62414693f951ddde0b","url":"#b-d4191c62414693f951ddde0b","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d15e6926903f560fa547d019","type":"p","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-d15e6926903f560fa547d019","content":"预测：只看问题，三个任务的保留集最佳常量回答分别约为 1/2、1/3、1/2，平均为 4/9≈44.4%。这比把七种答案混在一起说“随机率 1/7”更符合任务结构。","sha256":"25a797783c0d33246a29e6b05de2a88236b882aa0783b1729ebf8616970cba5d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9e609bf96c4e93aa06e026da","nextBlock":"b-a962abd04c046a7c528efcd3","prerequisites":[{"id":"b-8fe61110e25c104b4f431314","url":"#b-8fe61110e25c104b4f431314","role":"section-title"},{"id":"b-d4191c62414693f951ddde0b","url":"#b-d4191c62414693f951ddde0b","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a962abd04c046a7c528efcd3","type":"pre","scope":"block-ef7e95916f594784b1269ebf22f89911","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-a962abd04c046a7c528efcd3","content":"```\ndef draw_scene(count,left_color,right_shape,rng):\n    img=Image.new('RGB',(40,32),(244,242,236));d=ImageDraw.Draw(img)\n    slots=[10] if count==1 else [10,30]\n    for j,cx in enumerate(slots):\n        cy=16+int(rng.integers(-3,4));r=int(rng.integers(4,7));cx+=int(rng.integers(-1,2))\n        color=left_color if j==0 else str(rng.choice(list(COLORS)))\n        shape=right_shape if j==len(slots)-1 else str(rng.choice(['square','circle']))\n        box=(cx-r,cy-r,cx+r,cy+r)\n        if shape=='square':d.rectangle(box,fill=COLORS[color])\n        else:d.ellipse(box,fill=COLORS[color])\n    arr=np.array(img,dtype=np.float32)/255\n    return np.clip(arr+rng.normal(0,.014,arr.shape),0,1).astype(np.float32)\n\ndef make_vqa(n,seed,biased):\n    rng=np.random.default_rng(seed);images=[];answers=[];meta=[]\n    # Training has a 75% red prior. Test covers all 12 condition combinations equally.\n    combos=[(c,col,sh) for c in [1,2] for col in COLORS for sh in ['square','circle']]\n    for i in range(n):\n        if biased:\n            count=int(rng.choice([1,2]));color=str(rng.choice(list(COLORS),p=[.75,.125,.125]));shape=str(rng.choice(['square','circle']))\n        else:count,color,shape=combos[i%len(combos)]\n        images.append(draw_scene(count,color,shape,rng))\n        answers.append([A['one' if count==1 else 'two'],A[color],A[shape]])\n        meta.append({'count':count,'left_color':color,'right_shape':shape})\n    return torch.tensor(np.stack(images)).permute(0,3,1,2),torch.tensor(answers),meta\nXtr,Ytr,train_meta=make_vqa(720,14031,True)\nXte,Yte,test_meta=make_vqa(180,14032,False)\ntrain_hashes={hashlib.sha256(x.numpy().tobytes()).hexdigest() for x in Xtr}\nassert not train_hashes.intersection(hashlib.sha256(x.numpy().tobytes()).hexdigest() for x in Xte)\n# Metadata remains only in this data/label construction and plot captions.\n# model.forward receives tensors (pixels, question_id), never these dictionaries.\nnp.savez_compressed(OUT/'lab14-data.npz',train_images=Xtr.numpy(),train_answers=Ytr.numpy(),test_images=Xte.numpy(),test_answers=Yte.numpy())\nprint('train/test image counts:',len(Xtr),len(Xte),'pairs:',Ytr.numel(),Yte.numel())\nprint('train left-color prior:',{c:sum(m['left_color']==c for m in train_meta)/len(train_meta) for c in COLORS})\nprint('test left-color prior:',{c:sum(m['left_color']==c for m in test_meta)/len(test_meta) for c in COLORS})\n```","sha256":"36c815acc00c365f774968586f380e1fed0fcacf2b8ed3f268abf3b2845d3299","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d15e6926903f560fa547d019","nextBlock":null,"prerequisites":[{"id":"b-8fe61110e25c104b4f431314","url":"#b-8fe61110e25c104b4f431314","role":"section-title"},{"id":"b-d4191c62414693f951ddde0b","url":"#b-d4191c62414693f951ddde0b","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8824392af1bda1df08d061ac","type":"h2","scope":"block-62876c0daff44e6f9b582bcaa0d0dd00","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-8824392af1bda1df08d061ac","content":"## 3. 模型、损失与公平比较范围","sha256":"3129d617c9d64f12477cc585d4a415fb35853a79d91697ee3a67e21cf439afc7","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-6c6dcbedc229d455a09a9136","prerequisites":[{"id":"b-6c6dcbedc229d455a09a9136","url":"#b-6c6dcbedc229d455a09a9136","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-6c6dcbedc229d455a09a9136","type":"p","scope":"block-62876c0daff44e6f9b582bcaa0d0dd00","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-6c6dcbedc229d455a09a9136","content":"像素分支是两层 CNN 与保留布局的全连接表示；问题分支是 12 维可学习嵌入。拼接后输出七个答案的 logits，以交叉熵 $-\\log p(a\\mid I,q)$ 训练。question-only 使用相同问题嵌入维度、答案空间与训练样本，但省去 CNN。两者都真正更新参数，不能把手填“red”冒充训练的语言基线。","sha256":"bd1543535de10e14989b8f47346a883f97b200294db444f7b421779ebcfdcd67","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"-\\log p(a\\mid I,q)","definitionContext":"3. 模型、损失与公平比较范围"}],"previousBlock":"b-8824392af1bda1df08d061ac","nextBlock":"b-1059308191d5c884039d53ec","prerequisites":[{"id":"b-8824392af1bda1df08d061ac","url":"#b-8824392af1bda1df08d061ac","role":"section-title"}],"relatedBlocks":[]},{"id":"b-1059308191d5c884039d53ec","type":"p","scope":"block-62876c0daff44e6f9b582bcaa0d0dd00","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-1059308191d5c884039d53ec","content":"完整模型参数更多，因此二者差距本身不是严格的等参数容量结论。我们在同一个完整模型上又做像素置零与错配，固定参数再移除正确图像条件。该消融更直接检验本次预测是否使用图像。","sha256":"8eef598675a5a3470afd723119d407b25beb4b74308e2af413c81ebdca7c6581","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6c6dcbedc229d455a09a9136","nextBlock":"b-a4cb3ff6602b195295b683a6","prerequisites":[{"id":"b-8824392af1bda1df08d061ac","url":"#b-8824392af1bda1df08d061ac","role":"section-title"},{"id":"b-6c6dcbedc229d455a09a9136","url":"#b-6c6dcbedc229d455a09a9136","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a4cb3ff6602b195295b683a6","type":"pre","scope":"block-7f07825ae8a84326b6a7a768b26c7caf","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-a4cb3ff6602b195295b683a6","content":"```\nclass VQA(nn.Module):\n    def __init__(self,visual):\n        super().__init__();self.visual=visual\n        self.question=nn.Embedding(len(QUESTIONS),12)\n        if visual:\n            self.cnn=nn.Sequential(nn.Conv2d(3,12,3,padding=1),nn.ReLU(),nn.MaxPool2d(2),nn.Conv2d(12,24,3,padding=1),nn.ReLU(),nn.MaxPool2d(2),nn.Flatten(),nn.Linear(24*8*10,64),nn.ReLU())\n        self.answer=nn.Sequential(nn.Linear(64+12 if visual else 12,64),nn.ReLU(),nn.Linear(64,len(ANSWERS)))\n    def forward(self,pixels,question_id):\n        q=self.question(question_id)\n        features=torch.cat([self.cnn(pixels),q],-1) if self.visual else q\n        return self.answer(features)\n\nfull=VQA(True);question_only=VQA(False)\ncurve=[]\nfor label,net in [('image+question',full),('question-only',question_only)]:\n    optimizer=torch.optim.Adam(net.parameters(),lr=.002)\n    for epoch in range(28):\n        order=torch.randperm(Ytr.numel());net.train();loss_sum=0.\n        for pair_ids in order.split(72):\n            image_ids=pair_ids//3;qids=pair_ids%3\n            logits=net(Xtr[image_ids],qids);target=Ytr[image_ids,qids]\n            loss=F.cross_entropy(logits,target)\n            optimizer.zero_grad();loss.backward();optimizer.step();loss_sum+=float(loss.detach())*len(pair_ids)\n        if epoch in [0,5,13,27]:print(label,'epoch',epoch+1,'CE',round(loss_sum/Ytr.numel(),4))\n        curve.append({'model':label,'epoch':epoch+1,'train_ce':loss_sum/Ytr.numel()})\n    torch.save(net.state_dict(),OUT/('lab14-'+('visual' if net.visual else 'question-only')+'-weights.pt'))\n```","sha256":"701155870a1e8d1c9076048a018c3eac278a5c5ce5d1530da54c7367d1727177","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-1059308191d5c884039d53ec","nextBlock":null,"prerequisites":[{"id":"b-8824392af1bda1df08d061ac","url":"#b-8824392af1bda1df08d061ac","role":"section-title"},{"id":"b-6c6dcbedc229d455a09a9136","url":"#b-6c6dcbedc229d455a09a9136","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-18be02fd14ea8f6ad6adc9d2","type":"h2","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-18be02fd14ea8f6ad6adc9d2","content":"## 4. 逐问题读结果，别只看总准确率","sha256":"1fe68838bac1f3e94024769f31316795734a73b8163daa4afe9dffeb19a16c42","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-5a4380a2ba607f596eb8bdd9","prerequisites":[{"id":"b-5e7abc5bad9c9d07627a1294","url":"#b-5e7abc5bad9c9d07627a1294","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-5a4380a2ba607f596eb8bdd9","type":"table","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-5a4380a2ba607f596eb8bdd9","content":"| 保留集条件 | 数量 | 左侧颜色 | 右侧形状 | 全部 |\n| --- | --- | --- | --- | --- |\n| Image + question | 100.0% | 100.0% | 100.0% | 100.0% |\n| Question-only | 50.0% | 33.3% | 50.0% | 44.4% |\n| 同一模型，像素置零 | 50.0% | 33.3% | 50.0% | 44.4% |\n| 同一模型，图像循环错配 | 83.3% | 50.0% | 0.0% | 44.4% |","sha256":"c973352dfecd3c1e7c8a26ed089a8951087a83f682f227b3de93c8dcc5168934","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["保留集条件","数量","左侧颜色","右侧形状","全部"],"rows":[["保留集条件","数量","左侧颜色","右侧形状","全部"],["Image + question","100.0%","100.0%","100.0%","100.0%"],["Question-only","50.0%","33.3%","50.0%","44.4%"],["同一模型，像素置零","50.0%","33.3%","50.0%","44.4%"],["同一模型，图像循环错配","83.3%","50.0%","0.0%","44.4%"]]},"previousBlock":"b-18be02fd14ea8f6ad6adc9d2","nextBlock":"b-5e7abc5bad9c9d07627a1294","prerequisites":[{"id":"b-18be02fd14ea8f6ad6adc9d2","url":"#b-18be02fd14ea8f6ad6adc9d2","role":"section-title"},{"id":"b-5e7abc5bad9c9d07627a1294","url":"#b-5e7abc5bad9c9d07627a1294","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-5e7abc5bad9c9d07627a1294","type":"p","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-5e7abc5bad9c9d07627a1294","content":"循环错配不是各属性独立重抽：数据按组合排列，所以相邻样本常保留数量，却总改变形状。于是 83.3%/50%/0% 不是异常；它正提醒我们干预协议必须说清。不要把这个排列解释为各任务的随机机会水平。","sha256":"5772517b1201f9efe13bb04cb40965793bbfb2044defb920a18a4690ca93b8af","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-5a4380a2ba607f596eb8bdd9","nextBlock":"b-ffec4b0439146f71b12329c6","prerequisites":[{"id":"b-18be02fd14ea8f6ad6adc9d2","url":"#b-18be02fd14ea8f6ad6adc9d2","role":"section-title"}],"relatedBlocks":[]},{"id":"b-ffec4b0439146f71b12329c6","type":"p","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-ffec4b0439146f71b12329c6","content":"![相同保留问答上的视觉、语言、零图像与错配图像比较](/assets/display/fe08ead67458f0103fca897b.webp)","sha256":"96dafa969bed5d9ad470f33155f80bcacac14d744136bb6b734389f81e9e5cf9","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-5e7abc5bad9c9d07627a1294","nextBlock":"b-3b7226e13debda82f9b6a776","prerequisites":[{"id":"b-18be02fd14ea8f6ad6adc9d2","url":"#b-18be02fd14ea8f6ad6adc9d2","role":"section-title"},{"id":"b-5e7abc5bad9c9d07627a1294","url":"#b-5e7abc5bad9c9d07627a1294","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3b7226e13debda82f9b6a776","type":"p","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-3b7226e13debda82f9b6a776","content":"同图换问展示问题控制“读什么”，同问换图展示视觉控制“答什么”。先查控制图中红/蓝只变颜色的一对，再看代码记录的九条问答，判断本次证据是否同时满足这两个方向。","sha256":"c51336f20e1d778413d93dfee9689f8717a4228ddc7f0a4593271786481d265b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ffec4b0439146f71b12329c6","nextBlock":"b-fb0a6b5386c1ffc96594c11b","prerequisites":[{"id":"b-18be02fd14ea8f6ad6adc9d2","url":"#b-18be02fd14ea8f6ad6adc9d2","role":"section-title"},{"id":"b-5e7abc5bad9c9d07627a1294","url":"#b-5e7abc5bad9c9d07627a1294","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-fb0a6b5386c1ffc96594c11b","type":"pre","scope":"block-3ce456f62e2a4489be2b14a56d1b568e","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-fb0a6b5386c1ffc96594c11b","content":"```\n@torch.no_grad()\ndef answer_all(net,images):\n    net.eval()\n    return torch.stack([net(images,torch.full((len(images),),q)).argmax(-1) for q in range(3)],1)\nnormal=answer_all(full,Xte);qo=answer_all(question_only,Xte)\nzero=answer_all(full,torch.zeros_like(Xte));shuffled=answer_all(full,Xte[torch.roll(torch.arange(len(Xte)),1)])\ntrain_full=answer_all(full,Xtr);train_qo=answer_all(question_only,Xtr)\ndef score(p,y):return {'overall':float(p.eq(y).float().mean()),**{QUESTIONS[q]:float(p[:,q].eq(y[:,q]).float().mean()) for q in range(3)}}\nmetrics={'test_image_question':score(normal,Yte),'test_question_only':score(qo,Yte),'test_zero_pixels':score(zero,Yte),'test_shuffled_pixels':score(shuffled,Yte),'train_image_question':score(train_full,Ytr),'train_question_only':score(train_qo,Ytr)}\nprint(json.dumps(metrics,indent=2))\n# Recreate two images with matched nuisance randomness, only left color differs.\ncontrol_specs=[(2,'red','square'),(2,'blue','square'),(1,'green','circle')]\ncontrols_x=torch.tensor(np.stack([draw_scene(*s,np.random.default_rng(444)) for s in control_specs])).permute(0,3,1,2)\ncontrols_pred=answer_all(full,controls_x);controls_qo=answer_all(question_only,controls_x)\ncontrols=[{'image_condition':list(s),'question':q,'image_question':ANSWERS[int(controls_pred[i,j])],'question_only':ANSWERS[int(controls_qo[i,j])]} for i,s in enumerate(control_specs) for j,q in enumerate(QUESTIONS)]\nprint('same image / changed question and same question / changed image:')\nfor row in controls:print(row)\n```","sha256":"8e18a29d7fa6b2a327c610746be654c9657a1db37fbbcdf87d48868eec182c2d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3b7226e13debda82f9b6a776","nextBlock":null,"prerequisites":[{"id":"b-18be02fd14ea8f6ad6adc9d2","url":"#b-18be02fd14ea8f6ad6adc9d2","role":"section-title"},{"id":"b-5e7abc5bad9c9d07627a1294","url":"#b-5e7abc5bad9c9d07627a1294","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a9f9b9ab7bb4a4f29b95d3e2","type":"h2","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-a9f9b9ab7bb4a4f29b95d3e2","content":"## 5. 学习曲线与可以被推翻的结论","sha256":"1ca0d2332e22d968ff90ecec13d4503cd081f51138359dfa0245b7402b49375f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-2954f6eb6b4876cee8d0a83e","prerequisites":[{"id":"b-2954f6eb6b4876cee8d0a83e","url":"#b-2954f6eb6b4876cee8d0a83e","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2954f6eb6b4876cee8d0a83e","type":"p","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-2954f6eb6b4876cee8d0a83e","content":"![图像问答与只读问题模型的实际训练交叉熵](/assets/display/d154401bc0457c19795d01aa.webp)","sha256":"ef326977eb3c2d61ef5d4269aeb7b71d502b9f00abd948c34e3a83fd1eff4bba","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a9f9b9ab7bb4a4f29b95d3e2","nextBlock":"b-736d22f1f6ed4a061495a2c3","prerequisites":[{"id":"b-a9f9b9ab7bb4a4f29b95d3e2","url":"#b-a9f9b9ab7bb4a4f29b95d3e2","role":"section-title"}],"relatedBlocks":[]},{"id":"b-736d22f1f6ed4a061495a2c3","type":"p","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-736d22f1f6ed4a061495a2c3","content":"完整模型在这个有限合成分布上达到 100%，不等于解决真实 VQA。颜色、边缘和对象数都容易辨别，问题模板也固定。只改一个条件：保持图像不变，把未知改写“which hue is on the left?”输入当前 ID 接口会发生什么？正确结论是它根本没有这个文本编码能力，不应把手工映射改写到 ID 的功能归功于网络。","sha256":"beea7d2078e665fc442333a2e15bb69694a025ec8f0b7eb6e2db5b2861eb3a42","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2954f6eb6b4876cee8d0a83e","nextBlock":"b-f791f41befe36614964c608e","prerequisites":[{"id":"b-a9f9b9ab7bb4a4f29b95d3e2","url":"#b-a9f9b9ab7bb4a4f29b95d3e2","role":"section-title"},{"id":"b-2954f6eb6b4876cee8d0a83e","url":"#b-2954f6eb6b4876cee8d0a83e","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-f791f41befe36614964c608e","type":"p","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-f791f41befe36614964c608e","content":"另一项可运行改动是把训练颜色先验改为均匀，保留模型与测试不变。先预测 question-only 的训练颜色分数会下降，而保留集颜色分数仍约为三分之一。模型可以减少训练与测试的先验落差，却仍需要看图才能逐例回答。","sha256":"0632888c3952b95db1c950e8a9167e33450ef30280bad500214018db5eedcf4b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-736d22f1f6ed4a061495a2c3","nextBlock":"b-8e42f91101cbaab36fcc84cd","prerequisites":[{"id":"b-a9f9b9ab7bb4a4f29b95d3e2","url":"#b-a9f9b9ab7bb4a4f29b95d3e2","role":"section-title"},{"id":"b-2954f6eb6b4876cee8d0a83e","url":"#b-2954f6eb6b4876cee8d0a83e","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8e42f91101cbaab36fcc84cd","type":"pre","scope":"block-040689f0e3594ff988d47a70c68dbab6","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-8e42f91101cbaab36fcc84cd","content":"```\nfig,axes=plt.subplots(1,3,figsize=(11,4))\nfor i,ax in enumerate(axes):\n    ax.imshow(controls_x[i].permute(1,2,0));ax.axis('off')\n    rows=['count: ','left color: ','right shape: ']\n    text='\\n'.join(rows[j]+ANSWERS[int(controls_pred[i,j])]+' [Q-only: '+ANSWERS[int(controls_qo[i,j])]+']' for j in range(3))\n    ax.set_title(text,fontsize=10)\nfig.suptitle('LAB14: same questions, changed image; three questions per image',fontsize=13)\nfig.tight_layout();fig.savefig(OUT/'lab14-vqa-controls.png',dpi=160);plt.show();plt.close(fig)\nfig,ax=plt.subplots(figsize=(10,3.6));labels=['image + Q','Q only','zero pixels','shuffled pixels'];keys=['test_image_question','test_question_only','test_zero_pixels','test_shuffled_pixels'];values=[metrics[k]['overall'] for k in keys]\nax.bar(labels,values,color=['#356b91','#ad8251','#9da6ad','#9da6ad']);ax.set(ylim=(0,1.1),ylabel='answer accuracy (540 held-out pairs)')\nfor i,v in enumerate(values):ax.text(i,v+.02,f'{v:.1%}',ha='center')\nfig.tight_layout();fig.savefig(OUT/'lab14-vqa-ablation.png',dpi=160);plt.show();plt.close(fig)\nfig,ax=plt.subplots(figsize=(8,3.2))\nfor label in ['image+question','question-only']:\n    r=[v for v in curve if v['model']==label];ax.plot([v['epoch'] for v in r],[v['train_ce'] for v in r],label=label)\nax.set(xlabel='epoch',ylabel='answer cross entropy');ax.legend();fig.tight_layout();fig.savefig(OUT/'lab14-vqa-learning.png',dpi=160);plt.show();plt.close(fig)\nresult={'notebook_id':'11d9f74871544a77b42024a9fe06b81a','execution':'local CPU, fresh process; not a Deepnote cloud run','created_utc':datetime.now(timezone.utc).isoformat(),'elapsed_seconds':time.perf_counter()-START,'environment':{'python':sys.version,'torch':torch.__version__,'numpy':np.__version__,'platform':platform.platform()},'split':{'train_seed':14031,'test_seed':14032,'train_images':720,'test_images':180,'questions_per_image':3,'split_before_question_expansion':True,'train_red_prior':.75,'test_color_prior':'uniform by construction'},'metrics':metrics,'curve':curve,'controls':controls,'predictions':[{'truth':[ANSWERS[int(v)] for v in Yte[i]],'image_question':[ANSWERS[int(v)] for v in normal[i]],'question_only':[ANSWERS[int(v)] for v in qo[i]]} for i in range(len(Xte))],'files_sha256':{p.name:hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(OUT.glob('*.pt'))},'data_sha256':hashlib.sha256((OUT/'lab14-data.npz').read_bytes()).hexdigest()}\n(OUT/'lab14-results.json').write_text(json.dumps(result,indent=2));print('saved',OUT,'seconds',round(result['elapsed_seconds'],2))\n```","sha256":"2e78f928663ed19a4abe9bb865f2ec5b12eca912e29834ec3865f05c50ab3b0b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f791f41befe36614964c608e","nextBlock":null,"prerequisites":[{"id":"b-a9f9b9ab7bb4a4f29b95d3e2","url":"#b-a9f9b9ab7bb4a4f29b95d3e2","role":"section-title"},{"id":"b-2954f6eb6b4876cee8d0a83e","url":"#b-2954f6eb6b4876cee8d0a83e","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-701e3fb20f18c27420592e60","type":"h2","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-701e3fb20f18c27420592e60","content":"## 结果文件、来源与回链","sha256":"8660b9343b7f1d52be6bcc92613183fa8e5476cd47e6bec2381d18175cddfcca","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-0931a8ad2f714e26836deb30","prerequisites":[{"id":"b-0931a8ad2f714e26836deb30","url":"#b-0931a8ad2f714e26836deb30","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0931a8ad2f714e26836deb30","type":"p","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-0931a8ad2f714e26836deb30","content":"`lab14-data.npz` 保存像素与答案；两个 `.pt` 文件对应两个真实训练模型；`lab14-results.json` 保存每类准确率、逐轮损失、九条受控问答、逐图保留预测及环境/数据哈希。场景元数据只用于监督标签与解读，未传入网络。","sha256":"be9dcfab95622299ac41f14246ffc693f6f04b7a84339f461ae3821b17c802ec","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-701e3fb20f18c27420592e60","nextBlock":"b-ecb0f2afeb1daf83541dd0a4","prerequisites":[{"id":"b-701e3fb20f18c27420592e60","url":"#b-701e3fb20f18c27420592e60","role":"section-title"}],"relatedBlocks":[]},{"id":"b-ecb0f2afeb1daf83541dd0a4","type":"p","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-ecb0f2afeb1daf83541dd0a4","content":"[VQA 原论文，作者 arXiv 页面](https://arxiv.org/abs/1505.00468) 给出图像与问题到答案的任务定义与基线比较。[VQA 官方项目](https://visualqa.org/) 提供真实数据、评价与后续 VQA v2；本实验的精确分类准确率不是官方多人答案共识评分。我们的合成颜色先验是教学构造，不是该数据集的实测统计。","sha256":"605e525420770ca13650761ad0f9b0d9e751f8f5a21f8e29f246e4562f2f4a9e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0931a8ad2f714e26836deb30","nextBlock":"b-e552553b9a0a852b47ca618f","prerequisites":[{"id":"b-701e3fb20f18c27420592e60","url":"#b-701e3fb20f18c27420592e60","role":"section-title"},{"id":"b-0931a8ad2f714e26836deb30","url":"#b-0931a8ad2f714e26836deb30","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-e552553b9a0a852b47ca618f","type":"p","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/notebooks/11d9f74871544a77b42024a9fe06b81a.html#b-e552553b9a0a852b47ca618f","content":"回到 [CLS 03](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/680da245145447dd88983aabd136fd24)：CLIP 排候选描述，caption 自回归生成，VQA 在问题条件下输出答案。它们共享视觉与语言，输出空间和训练目标却不同，不能因为界面都接受文字就互相替代。","sha256":"4bdd15d61276026eb047e031a5db6e0226050b63c5c87eeb2a31ae52cb226668","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"language-local-and-question","title":"语言怎样选择当前需要的视觉信息？"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ecb0f2afeb1daf83541dd0a4","nextBlock":null,"prerequisites":[{"id":"b-701e3fb20f18c27420592e60","url":"#b-701e3fb20f18c27420592e60","role":"section-title"},{"id":"b-0931a8ad2f714e26836deb30","url":"#b-0931a8ad2f714e26836deb30","role":"problem-context"}],"relatedBlocks":[]}]}