{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"LAB 14 · 图像问答：视觉条件与问题先验","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html","markdown":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.md","pages":[{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":1},{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":2},{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":3},{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":4},{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":5},{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":6},{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":7},{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","unit":"LAB 14 · 图像问答：视觉条件与问题先验","unitId":"lab14-视觉问答与语言捷径","chapter":"course","number":8}],"aliases":[{"oldId":"b-e8a9fa91421744b0b547ae04","targetId":"b-e8a9fa91421744b0b547ae04","page":"reading-opening","operation":"move","beforeSHA256":"1f37c330374145c33cdc473dedf57906452b35f19f323be695280cb12f17875b","afterSHA256":"1f37c330374145c33cdc473dedf57906452b35f19f323be695280cb12f17875b","reason":"Same source body in authored semantic page"},{"oldId":"b-0e8e7ed2e6de87158f4cd3a9","targetId":"b-0e8e7ed2e6de87158f4cd3a9","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-7215d08065fd4e0b05ece0a9","targetId":"b-7215d08065fd4e0b05ece0a9","page":"reading-opening","operation":"move","beforeSHA256":"5790b0afb52508f0174b1bc581f7847035a51c5c2a46e5c821bcd9d5d61bcbb0","afterSHA256":"5790b0afb52508f0174b1bc581f7847035a51c5c2a46e5c821bcd9d5d61bcbb0","reason":"Same source body in authored semantic page"},{"oldId":"b-273322267152a728e8249259","targetId":"b-273322267152a728e8249259","page":"reading-opening","operation":"move","beforeSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","afterSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","reason":"Same source body in authored semantic page"},{"oldId":"b-56c2d0d67f5820bcf11b2122","targetId":"b-56c2d0d67f5820bcf11b2122","page":"reading-opening","operation":"move","beforeSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","afterSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","reason":"Same source body in authored semantic page"},{"oldId":"b-62b443c3d41d2508b5f26a04","targetId":"b-62b443c3d41d2508b5f26a04","page":"reading-opening","operation":"move","beforeSHA256":"54bacb112e17b840a5a310ce968bdae21f53376a4ba54a33e651576ea0a41a11","afterSHA256":"54bacb112e17b840a5a310ce968bdae21f53376a4ba54a33e651576ea0a41a11","reason":"Same source body in authored semantic page"},{"oldId":"b-fc4680bd358f7240377a90b7","targetId":"b-fc4680bd358f7240377a90b7","page":"reading-opening","operation":"move","beforeSHA256":"a39bb3bfff571024e21cbf171c189671131c82900779f89b74b1609b897f31b1","afterSHA256":"a39bb3bfff571024e21cbf171c189671131c82900779f89b74b1609b897f31b1","reason":"Same source body in authored semantic page"},{"oldId":"b-703bec265f6342ea64adfeec","targetId":"b-703bec265f6342ea64adfeec","page":"reading-opening","operation":"move","beforeSHA256":"4790c4b243b6afc33a3b0a94b904dcef5a39431db1ef3a267d3dec7d13a4df01","afterSHA256":"4790c4b243b6afc33a3b0a94b904dcef5a39431db1ef3a267d3dec7d13a4df01","reason":"Same source body in authored semantic page"},{"oldId":"b-42b447ddf794e989ac8744f9","targetId":"b-42b447ddf794e989ac8744f9","page":"reading-opening","operation":"move","beforeSHA256":"a2c341314db4e1d785717eeb1afabfbccce760d49244c5ce15bae05cabd5117a","afterSHA256":"9a2655d1601e5f06db61c1f983135287b33ee7b26e382647125686614f100163","reason":"Same source body in authored semantic page"},{"oldId":"b-a5e5a78ae1c10074c0f54c1a","targetId":"b-a5e5a78ae1c10074c0f54c1a","page":"reading-opening","operation":"move","beforeSHA256":"36843086e2b941e4d4724082772339fb7dc1f03fc1e5596c7f355166ed31e988","afterSHA256":"36843086e2b941e4d4724082772339fb7dc1f03fc1e5596c7f355166ed31e988","reason":"Same source body in authored semantic page"},{"oldId":"b-70c93de6ea60cc57b1a6b09b","targetId":"b-70c93de6ea60cc57b1a6b09b","page":"reading-block-4919dea7730a4e19823525d3b77d89c9","operation":"move","beforeSHA256":"8561dd038e5430bf28bca9be060c043b2540da69b265d2984f130ede699934a5","afterSHA256":"8561dd038e5430bf28bca9be060c043b2540da69b265d2984f130ede699934a5","reason":"Same source body in authored semantic page"},{"oldId":"b-a08ad84cf4047aa85dafb2cc","targetId":"b-a08ad84cf4047aa85dafb2cc","page":"reading-block-4919dea7730a4e19823525d3b77d89c9","operation":"move","beforeSHA256":"35402536e21a2aa99b1809d10824524d03cd182909456a61a350c6b6aa2aad8f","afterSHA256":"35402536e21a2aa99b1809d10824524d03cd182909456a61a350c6b6aa2aad8f","reason":"Same source body in authored semantic page"},{"oldId":"b-97bdba7dababb81b831c6338","targetId":"b-97bdba7dababb81b831c6338","page":"reading-block-4919dea7730a4e19823525d3b77d89c9","operation":"move","beforeSHA256":"c87a6ce586309b464659eaf822eb0ff4d176127095e5ad36a39eed58c5928689","afterSHA256":"c87a6ce586309b464659eaf822eb0ff4d176127095e5ad36a39eed58c5928689","reason":"Same source body in authored semantic page"},{"oldId":"b-a798cda734228a50dbcfc2cd","targetId":"b-a798cda734228a50dbcfc2cd","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"b6acdcaa9ced1ec6dd65e6de6981dbcd628d7bfd828f0f12c62297e0630bc40c","afterSHA256":"b6acdcaa9ced1ec6dd65e6de6981dbcd628d7bfd828f0f12c62297e0630bc40c","reason":"Same source body in authored semantic page"},{"oldId":"b-cadaa94ad744638f889c4ced","targetId":"b-cadaa94ad744638f889c4ced","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"23b2824dee5d302542c4d7c618677b4934afb306b67f64f4ca576b3065f2a57f","afterSHA256":"23b2824dee5d302542c4d7c618677b4934afb306b67f64f4ca576b3065f2a57f","reason":"Same source body in authored semantic page"},{"oldId":"b-78d9b3f45c94d4f95cd4c476","targetId":"b-78d9b3f45c94d4f95cd4c476","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"86d6ce3d14d4990ac0f81c69516fc4d095213a1a3b4633f5d8aa2e0196bc90fd","afterSHA256":"86d6ce3d14d4990ac0f81c69516fc4d095213a1a3b4633f5d8aa2e0196bc90fd","reason":"Same source body in authored semantic page"},{"oldId":"b-c29be401a5996bcae09fb7dc","targetId":"b-c29be401a5996bcae09fb7dc","page":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","operation":"move","beforeSHA256":"810908fd69927093502edf9303e0f30160207a520fff2e154f31763f77c1dc30","afterSHA256":"810908fd69927093502edf9303e0f30160207a520fff2e154f31763f77c1dc30","reason":"Same source body in authored semantic page"},{"oldId":"b-111ba4b937bba57f308b5cc8","targetId":"b-111ba4b937bba57f308b5cc8","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"3755eb55c18436f73604f02646360ddd44f5e051c982148d4ea071c8165155ea","afterSHA256":"3755eb55c18436f73604f02646360ddd44f5e051c982148d4ea071c8165155ea","reason":"Same source body in authored semantic page"},{"oldId":"b-bc13c340b943c002ea96aad1","targetId":"b-bc13c340b943c002ea96aad1","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"70bc354d398d992dadd4827ac6b23f9133c0a8c443756ad4656e7371c41512d5","afterSHA256":"70bc354d398d992dadd4827ac6b23f9133c0a8c443756ad4656e7371c41512d5","reason":"Same source body in authored semantic page"},{"oldId":"b-6c0a1352ca05871bfa05f262","targetId":"b-6c0a1352ca05871bfa05f262","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"4e4b94606681bb6d2eb7b4afda09881a8d3ee251bb26915bd8dab24867b299a2","afterSHA256":"4e4b94606681bb6d2eb7b4afda09881a8d3ee251bb26915bd8dab24867b299a2","reason":"Same source body in authored semantic page"},{"oldId":"b-784289b33e0961699486872d","targetId":"b-784289b33e0961699486872d","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"25a797783c0d33246a29e6b05de2a88236b882aa0783b1729ebf8616970cba5d","afterSHA256":"25a797783c0d33246a29e6b05de2a88236b882aa0783b1729ebf8616970cba5d","reason":"Same source body in authored semantic page"},{"oldId":"b-804742a6dd17730405f1c272","targetId":"b-804742a6dd17730405f1c272","page":"reading-block-20b57075949948ac8d6711cd4621f5b6","operation":"move","beforeSHA256":"36c815acc00c365f774968586f380e1fed0fcacf2b8ed3f268abf3b2845d3299","afterSHA256":"36c815acc00c365f774968586f380e1fed0fcacf2b8ed3f268abf3b2845d3299","reason":"Same source body in authored semantic page"},{"oldId":"b-f973e05d78936921ede3bc6e","targetId":"b-f973e05d78936921ede3bc6e","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"3129d617c9d64f12477cc585d4a415fb35853a79d91697ee3a67e21cf439afc7","afterSHA256":"3129d617c9d64f12477cc585d4a415fb35853a79d91697ee3a67e21cf439afc7","reason":"Same source body in authored semantic page"},{"oldId":"b-63535d833b2eabff2b29b66d","targetId":"b-63535d833b2eabff2b29b66d","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"bd1543535de10e14989b8f47346a883f97b200294db444f7b421779ebcfdcd67","afterSHA256":"bd1543535de10e14989b8f47346a883f97b200294db444f7b421779ebcfdcd67","reason":"Same source body in authored semantic page"},{"oldId":"b-8768a540b23aaca30ba96bf8","targetId":"b-8768a540b23aaca30ba96bf8","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"8eef598675a5a3470afd723119d407b25beb4b74308e2af413c81ebdca7c6581","afterSHA256":"8eef598675a5a3470afd723119d407b25beb4b74308e2af413c81ebdca7c6581","reason":"Same source body in authored semantic page"},{"oldId":"b-b4feadbbb801d4d4074bb762","targetId":"b-b4feadbbb801d4d4074bb762","page":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","operation":"move","beforeSHA256":"701155870a1e8d1c9076048a018c3eac278a5c5ce5d1530da54c7367d1727177","afterSHA256":"701155870a1e8d1c9076048a018c3eac278a5c5ce5d1530da54c7367d1727177","reason":"Same source body in authored semantic page"},{"oldId":"b-100c1206964361ca121d899e","targetId":"b-100c1206964361ca121d899e","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"1fe68838bac1f3e94024769f31316795734a73b8163daa4afe9dffeb19a16c42","afterSHA256":"1fe68838bac1f3e94024769f31316795734a73b8163daa4afe9dffeb19a16c42","reason":"Same source body in authored semantic page"},{"oldId":"b-fbbc153f2d4f0c8e493fede2","targetId":"b-fbbc153f2d4f0c8e493fede2","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"rewrite","beforeSHA256":"1a66cf7d55a82b0f7527859ddd0e5368c56f264de60399a8230e2de00d0acaba","afterSHA256":"c973352dfecd3c1e7c8a26ed089a8951087a83f682f227b3de93c8dcc5168934","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-4bccd162cb617dd51f18bb8f","targetId":"b-4bccd162cb617dd51f18bb8f","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"5772517b1201f9efe13bb04cb40965793bbfb2044defb920a18a4690ca93b8af","afterSHA256":"5772517b1201f9efe13bb04cb40965793bbfb2044defb920a18a4690ca93b8af","reason":"Same source body in authored semantic page"},{"oldId":"b-f8a807e341bc9ea4085373d6","targetId":"b-f8a807e341bc9ea4085373d6","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"af9db8f48afeeba125b9490230557bd0aa98c406e67197f8697291fd1cf52a14","afterSHA256":"96dafa969bed5d9ad470f33155f80bcacac14d744136bb6b734389f81e9e5cf9","reason":"Same source body in authored semantic page"},{"oldId":"b-4b1e2c58cda67c89133dfc1f","targetId":"b-4b1e2c58cda67c89133dfc1f","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"c51336f20e1d778413d93dfee9689f8717a4228ddc7f0a4593271786481d265b","afterSHA256":"c51336f20e1d778413d93dfee9689f8717a4228ddc7f0a4593271786481d265b","reason":"Same source body in authored semantic page"},{"oldId":"b-240e1c1b77fcc6c7c6898754","targetId":"b-240e1c1b77fcc6c7c6898754","page":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","operation":"move","beforeSHA256":"8e18a29d7fa6b2a327c610746be654c9657a1db37fbbcdf87d48868eec182c2d","afterSHA256":"8e18a29d7fa6b2a327c610746be654c9657a1db37fbbcdf87d48868eec182c2d","reason":"Same source body in authored semantic page"},{"oldId":"b-66c28521a295f98b98be5efc","targetId":"b-66c28521a295f98b98be5efc","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"1ca0d2332e22d968ff90ecec13d4503cd081f51138359dfa0245b7402b49375f","afterSHA256":"1ca0d2332e22d968ff90ecec13d4503cd081f51138359dfa0245b7402b49375f","reason":"Same source body in authored semantic page"},{"oldId":"b-26a733100939c30ca63aede5","targetId":"b-26a733100939c30ca63aede5","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"27958004cbb247516adb951070045ae59fb33da0aa36cc1ec01cdab2c166f0e0","afterSHA256":"ef326977eb3c2d61ef5d4269aeb7b71d502b9f00abd948c34e3a83fd1eff4bba","reason":"Same source body in authored semantic page"},{"oldId":"b-f34764783ef77c063ed37e11","targetId":"b-f34764783ef77c063ed37e11","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"beea7d2078e665fc442333a2e15bb69694a025ec8f0b7eb6e2db5b2861eb3a42","afterSHA256":"beea7d2078e665fc442333a2e15bb69694a025ec8f0b7eb6e2db5b2861eb3a42","reason":"Same source body in authored semantic page"},{"oldId":"b-5d36271c2a1005b62ee8f380","targetId":"b-5d36271c2a1005b62ee8f380","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"0632888c3952b95db1c950e8a9167e33450ef30280bad500214018db5eedcf4b","afterSHA256":"0632888c3952b95db1c950e8a9167e33450ef30280bad500214018db5eedcf4b","reason":"Same source body in authored semantic page"},{"oldId":"b-3a920f103a300341bebe33ce","targetId":"b-3a920f103a300341bebe33ce","page":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","operation":"move","beforeSHA256":"2e78f928663ed19a4abe9bb865f2ec5b12eca912e29834ec3865f05c50ab3b0b","afterSHA256":"2e78f928663ed19a4abe9bb865f2ec5b12eca912e29834ec3865f05c50ab3b0b","reason":"Same source body in authored semantic page"},{"oldId":"b-482d3ed9e12973d3e50394ac","targetId":"b-482d3ed9e12973d3e50394ac","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"8660b9343b7f1d52be6bcc92613183fa8e5476cd47e6bec2381d18175cddfcca","afterSHA256":"8660b9343b7f1d52be6bcc92613183fa8e5476cd47e6bec2381d18175cddfcca","reason":"Same source body in authored semantic page"},{"oldId":"b-ed259a07be23e60a9cce758e","targetId":"b-ed259a07be23e60a9cce758e","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"be9dcfab95622299ac41f14246ffc693f6f04b7a84339f461ae3821b17c802ec","afterSHA256":"be9dcfab95622299ac41f14246ffc693f6f04b7a84339f461ae3821b17c802ec","reason":"Same source body in authored semantic page"},{"oldId":"b-6934ca394df57a14baa917ee","targetId":"b-6934ca394df57a14baa917ee","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"605e525420770ca13650761ad0f9b0d9e751f8f5a21f8e29f246e4562f2f4a9e","afterSHA256":"605e525420770ca13650761ad0f9b0d9e751f8f5a21f8e29f246e4562f2f4a9e","reason":"Same source body in authored semantic page"},{"oldId":"b-19d3dd405f17209941df7d8f","targetId":"b-19d3dd405f17209941df7d8f","page":"reading-block-193cc04bc38242b7be21838002bf9285","operation":"move","beforeSHA256":"4bdd15d61276026eb047e031a5db6e0226050b63c5c87eeb2a31ae52cb226668","afterSHA256":"4bdd15d61276026eb047e031a5db6e0226050b63c5c87eeb2a31ae52cb226668","reason":"Same source body in authored semantic page"},{"oldId":"block-4919dea7730a4e19823525d3b77d89c9--0-先给原例子正确命名","targetId":"b-70c93de6ea60cc57b1a6b09b","operation":"retain-source-identity"},{"oldId":"block-3c10b348d7d1412b80d07fc5583e1a27--1-环境与有限问题接口","targetId":"b-a798cda734228a50dbcfc2cd","operation":"retain-source-identity"},{"oldId":"block-20b57075949948ac8d6711cd4621f5b6--2-先按图像切分避免同一图的不同问题泄漏","targetId":"b-111ba4b937bba57f308b5cc8","operation":"retain-source-identity"},{"oldId":"block-62876c0daff44e6f9b582bcaa0d0dd00--3-模型损失与公平比较范围","targetId":"b-f973e05d78936921ede3bc6e","operation":"retain-source-identity"},{"oldId":"block-4eb1163b8bc64dd5ab088d22e10318bb--4-逐问题读结果别只看总准确率","targetId":"b-100c1206964361ca121d899e","operation":"retain-source-identity"},{"oldId":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8--5-学习曲线与可以被推翻的结论","targetId":"b-66c28521a295f98b98be5efc","operation":"retain-source-identity"},{"oldId":"block-193cc04bc38242b7be21838002bf9285--结果文件来源与回链","targetId":"b-482d3ed9e12973d3e50394ac","operation":"retain-source-identity"}],"blocks":[{"id":"b-e8a9fa91421744b0b547ae04","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-e8a9fa91421744b0b547ae04","content":"# LAB 14 · 图像问答：视觉条件与问题先验","sha256":"1f37c330374145c33cdc473dedf57906452b35f19f323be695280cb12f17875b","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-0e8e7ed2e6de87158f4cd3a9","prerequisites":[{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0e8e7ed2e6de87158f4cd3a9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-0e8e7ed2e6de87158f4cd3a9","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e8a9fa91421744b0b547ae04","nextBlock":"b-7215d08065fd4e0b05ece0a9","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"}],"relatedBlocks":[]},{"id":"b-7215d08065fd4e0b05ece0a9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-7215d08065fd4e0b05ece0a9","content":"RGB+问题输入、question-only对照、同图换问与同问换图","sha256":"5790b0afb52508f0174b1bc581f7847035a51c5c2a46e5c821bcd9d5d61bcbb0","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0e8e7ed2e6de87158f4cd3a9","nextBlock":"b-273322267152a728e8249259","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-273322267152a728e8249259","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-273322267152a728e8249259","content":"可先阅读已有结果，再决定是否运行。重算时请核对各节记录的数据、参数、环境与日期。","sha256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7215d08065fd4e0b05ece0a9","nextBlock":"b-56c2d0d67f5820bcf11b2122","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-56c2d0d67f5820bcf11b2122","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-56c2d0d67f5820bcf11b2122","content":"**学习路径：** [图像怎样约束句子与回答](/course/study/caption-vqa.html)","sha256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-273322267152a728e8249259","nextBlock":"b-62b443c3d41d2508b5f26a04","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-62b443c3d41d2508b5f26a04","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-62b443c3d41d2508b5f26a04","content":"同图换问题，模型读什么会变；同问换图片，答案也应跟着变。先读[P54](/course/pages/l3-r2-p54.html)，再看第2节按图切分和第4节逐问题结果。","sha256":"54bacb112e17b840a5a310ce968bdae21f53376a4ba54a33e651576ea0a41a11","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-56c2d0d67f5820bcf11b2122","nextBlock":"b-fc4680bd358f7240377a90b7","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-fc4680bd358f7240377a90b7","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-fc4680bd358f7240377a90b7","content":"开头字典例是已知属性oracle，主体才是读取RGB像素与问题ID的CNN，并单独训练question-only。三类问题为数量、左色、右形，答案为有限词表；这不等于开放语言问答。[完整控制和证据](/course/blocks/vqa-pixel-evidence.html)。","sha256":"a39bb3bfff571024e21cbf171c189671131c82900779f89b74b1609b897f31b1","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-62b443c3d41d2508b5f26a04","nextBlock":"b-703bec265f6342ea64adfeec","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-703bec265f6342ea64adfeec","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-703bec265f6342ea64adfeec","content":"读完应能解释question-only为何约44.44%，并从同图换问/同问换图各拿一个例子支持“模型使用了条件”。只读可先看2026-09-22保存结果，运行则按全部代码顺序；本次整理不改历史成绩。若答案需要一个区域而非类别，继续[LAB 15](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/7dc83786d067465aa76264475db2c62d)。","sha256":"4790c4b243b6afc33a3b0a94b904dcef5a39431db1ef3a267d3dec7d13a4df01","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-fc4680bd358f7240377a90b7","nextBlock":"b-42b447ddf794e989ac8744f9","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-42b447ddf794e989ac8744f9","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-42b447ddf794e989ac8744f9","content":"![同图换问与同问换图](/assets/display/2f7b2168f5b21ee2095709fa.webp)","sha256":"9a2655d1601e5f06db61c1f983135287b33ee7b26e382647125686614f100163","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-703bec265f6342ea64adfeec","nextBlock":"b-a5e5a78ae1c10074c0f54c1a","prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a5e5a78ae1c10074c0f54c1a","type":"p","scope":"block-9efe941652bc4525b09d4348dcef8323","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-a5e5a78ae1c10074c0f54c1a","content":"前两幅共享形状、位置和随机噪声，只改左侧颜色；第三幅只用于展示另一个问题条件，不是与前两幅只改一项。想让答案直接成为一个区域，继续 [LAB 15 · 语言指代小练习：目标、参照与关系](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/7dc83786d067465aa76264475db2c62d?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=7dc83786d067465aa76264475db2c62d&utm_term=get_notebook)。","sha256":"36843086e2b941e4d4724082772339fb7dc1f03fc1e5596c7f355166ed31e988","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-opening","title":"LAB 14 · 图像问答：视觉条件与问题先验","number":1},"sectionTitle":"LAB 14 · 图像问答：视觉条件与问题先验","currentQuestion":"LAB 14 · 图像问答：视觉条件与问题先验","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-42b447ddf794e989ac8744f9","nextBlock":null,"prerequisites":[{"id":"b-e8a9fa91421744b0b547ae04","url":"#b-e8a9fa91421744b0b547ae04","role":"section-title"},{"id":"b-0e8e7ed2e6de87158f4cd3a9","url":"#b-0e8e7ed2e6de87158f4cd3a9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-70c93de6ea60cc57b1a6b09b","type":"h2","scope":"block-4919dea7730a4e19823525d3b77d89c9","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-70c93de6ea60cc57b1a6b09b","content":"## 0. 先给原例子正确命名","sha256":"8561dd038e5430bf28bca9be060c043b2540da69b265d2984f130ede699934a5","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","number":2},"sectionTitle":"0. 先给原例子正确命名","currentQuestion":"0. 先给原例子正确命名","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-a08ad84cf4047aa85dafb2cc","prerequisites":[{"id":"b-a08ad84cf4047aa85dafb2cc","url":"#b-a08ad84cf4047aa85dafb2cc","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a08ad84cf4047aa85dafb2cc","type":"p","scope":"block-4919dea7730a4e19823525d3b77d89c9","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-a08ad84cf4047aa85dafb2cc","content":"原 `with-image` 分支读取字典中的 `cups` 与 `left`。这些已是答案属性，因此此处保留为 **known-attribute oracle**。它能说明问题选择不同字段，却没有完成视觉感知。下面打印原例，再与真正从像素学习的模型区分。","sha256":"35402536e21a2aa99b1809d10824524d03cd182909456a61a350c6b6aa2aad8f","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","number":2},"sectionTitle":"0. 先给原例子正确命名","currentQuestion":"0. 先给原例子正确命名","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-70c93de6ea60cc57b1a6b09b","nextBlock":"b-97bdba7dababb81b831c6338","prerequisites":[{"id":"b-70c93de6ea60cc57b1a6b09b","url":"#b-70c93de6ea60cc57b1a6b09b","role":"section-title"}],"relatedBlocks":[]},{"id":"b-97bdba7dababb81b831c6338","type":"pre","scope":"block-ee681c33c81c4de98ecea27d981ddb73","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-97bdba7dababb81b831c6338","content":"```\n# Detached readonly runs write only to an isolated ephemeral directory.\nimport os, tempfile\n_LAB_WORKDIR = tempfile.mkdtemp(prefix=\"lecture03-lab14-\")\nos.chdir(_LAB_WORKDIR)\nprint(\"ephemeral workdir:\", _LAB_WORKDIR)\n\ncases=[{'cups':2,'left':'red','q':'count'},{'cups':2,'left':'red','q':'left-color'},{'cups':2,'left':'blue','q':'left-color'}]\nfor c in cases:\n    prior='2' if c['q']=='count' else 'red'\n    oracle=str(c['cups']) if c['q']=='count' else c['left']\n    print(c,'question-only rule',prior,'known-attribute oracle',oracle)\n# No image was read above. The learned visual path begins below.\n```","sha256":"c87a6ce586309b464659eaf822eb0ff4d176127095e5ad36a39eed58c5928689","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4919dea7730a4e19823525d3b77d89c9","title":"0. 先给原例子正确命名","number":2},"sectionTitle":"0. 先给原例子正确命名","currentQuestion":"0. 先给原例子正确命名","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a08ad84cf4047aa85dafb2cc","nextBlock":null,"prerequisites":[{"id":"b-70c93de6ea60cc57b1a6b09b","url":"#b-70c93de6ea60cc57b1a6b09b","role":"section-title"},{"id":"b-a08ad84cf4047aa85dafb2cc","url":"#b-a08ad84cf4047aa85dafb2cc","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a798cda734228a50dbcfc2cd","type":"h2","scope":"block-3c10b348d7d1412b80d07fc5583e1a27","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-a798cda734228a50dbcfc2cd","content":"## 1. 环境与有限问题接口","sha256":"b6acdcaa9ced1ec6dd65e6de6981dbcd628d7bfd828f0f12c62297e0630bc40c","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-cadaa94ad744638f889c4ced","prerequisites":[{"id":"b-cadaa94ad744638f889c4ced","url":"#b-cadaa94ad744638f889c4ced","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-cadaa94ad744638f889c4ced","type":"p","scope":"block-3c10b348d7d1412b80d07fc5583e1a27","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-cadaa94ad744638f889c4ced","content":"依赖 NumPy、Pillow、Matplotlib、PyTorch；Restart & Run All 后无需别册内存。输出保存在 `outputs/lab14/`。2026-09-22 本地 CPU 已执行，PyTorch 2.14.0+cpu，约十余秒；不代表 Deepnote 云端执行。","sha256":"23b2824dee5d302542c4d7c618677b4934afb306b67f64f4ca576b3065f2a57f","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a798cda734228a50dbcfc2cd","nextBlock":"b-78d9b3f45c94d4f95cd4c476","prerequisites":[{"id":"b-a798cda734228a50dbcfc2cd","url":"#b-a798cda734228a50dbcfc2cd","role":"section-title"}],"relatedBlocks":[]},{"id":"b-78d9b3f45c94d4f95cd4c476","type":"p","scope":"block-3c10b348d7d1412b80d07fc5583e1a27","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-78d9b3f45c94d4f95cd4c476","content":"我们定义三种英文模板，用 ID 0/1/2 输入嵌入层。文字是模板的可读形式，模型不支持任意自然语言改写。答案空间为 one、two、red、blue、green、square、circle。这是封闭词表 VQA，足以检验条件依赖，但不能证明开放问答、常识推理或语言理解。","sha256":"86d6ce3d14d4990ac0f81c69516fc4d095213a1a3b4633f5d8aa2e0196bc90fd","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-cadaa94ad744638f889c4ced","nextBlock":"b-c29be401a5996bcae09fb7dc","prerequisites":[{"id":"b-a798cda734228a50dbcfc2cd","url":"#b-a798cda734228a50dbcfc2cd","role":"section-title"},{"id":"b-cadaa94ad744638f889c4ced","url":"#b-cadaa94ad744638f889c4ced","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c29be401a5996bcae09fb7dc","type":"pre","scope":"block-924df89109464fed8b7817f3ebf2b233","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-c29be401a5996bcae09fb7dc","content":"```\nfrom pathlib import Path\nimport json, hashlib, platform, sys, time\nfrom datetime import datetime, timezone\nimport numpy as np\nfrom PIL import Image, ImageDraw\nimport matplotlib\nif 'get_ipython' not in globals(): matplotlib.use('Agg')\nimport matplotlib.pyplot as plt\nimport torch\nfrom torch import nn\nimport torch.nn.functional as F\nOUT=Path('outputs/lab14');OUT.mkdir(parents=True,exist_ok=True)\ntorch.set_num_threads(1);torch.manual_seed(1403);np.random.seed(1403);START=time.perf_counter()\nCOLORS={'red':(220,55,55),'blue':(45,95,225),'green':(35,170,75)}\nQUESTIONS=['how many objects?','what color is the leftmost object?','what shape is the rightmost object?']\nANSWERS=['one','two','red','blue','green','square','circle']; A={x:i for i,x in enumerate(ANSWERS)}\nprint('CPU',torch.__version__,'seed 1403; answer vocabulary:',ANSWERS)\n```","sha256":"810908fd69927093502edf9303e0f30160207a520fff2e154f31763f77c1dc30","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-3c10b348d7d1412b80d07fc5583e1a27","title":"1. 环境与有限问题接口","number":3},"sectionTitle":"1. 环境与有限问题接口","currentQuestion":"1. 环境与有限问题接口","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-78d9b3f45c94d4f95cd4c476","nextBlock":null,"prerequisites":[{"id":"b-a798cda734228a50dbcfc2cd","url":"#b-a798cda734228a50dbcfc2cd","role":"section-title"},{"id":"b-cadaa94ad744638f889c4ced","url":"#b-cadaa94ad744638f889c4ced","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-111ba4b937bba57f308b5cc8","type":"h2","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-111ba4b937bba57f308b5cc8","content":"## 2. 先按图像切分，避免同一图的不同问题泄漏","sha256":"3755eb55c18436f73604f02646360ddd44f5e051c982148d4ea071c8165155ea","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-bc13c340b943c002ea96aad1","prerequisites":[{"id":"b-bc13c340b943c002ea96aad1","url":"#b-bc13c340b943c002ea96aad1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-bc13c340b943c002ea96aad1","type":"p","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-bc13c340b943c002ea96aad1","content":"每张图含一个或两个带颜色的方形/圆形。所有图都提出同样三个问题：数量、最左物体颜色、最右物体形状；只有一个物体时，它同时是最左与最右。目标由生成器标注，这是监督数据的来源，不是推理接口。`forward(pixels,question_id)` 无法读取场景字典。","sha256":"70bc354d398d992dadd4827ac6b23f9133c0a8c443756ad4656e7371c41512d5","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-111ba4b937bba57f308b5cc8","nextBlock":"b-6c0a1352ca05871bfa05f262","prerequisites":[{"id":"b-111ba4b937bba57f308b5cc8","url":"#b-111ba4b937bba57f308b5cc8","role":"section-title"}],"relatedBlocks":[]},{"id":"b-6c0a1352ca05871bfa05f262","type":"p","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-6c0a1352ca05871bfa05f262","content":"训练图 seed 14031、保留图 seed 14032；先得到 720/180 张互不相同的图，再扩展为 2160/540 个问答。训练左侧颜色以 75% 概率采红，实际本次为 73.61%；保留集三色各占三分之一。数量与形状平衡设计下各有两种可能。语言模型因此能在训练颜色问题上靠红色先验得高分，却没有随图改变的证据。","sha256":"4e4b94606681bb6d2eb7b4afda09881a8d3ee251bb26915bd8dab24867b299a2","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-bc13c340b943c002ea96aad1","nextBlock":"b-784289b33e0961699486872d","prerequisites":[{"id":"b-111ba4b937bba57f308b5cc8","url":"#b-111ba4b937bba57f308b5cc8","role":"section-title"},{"id":"b-bc13c340b943c002ea96aad1","url":"#b-bc13c340b943c002ea96aad1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-784289b33e0961699486872d","type":"p","scope":"block-20b57075949948ac8d6711cd4621f5b6","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-784289b33e0961699486872d","content":"预测：只看问题，三个任务的保留集最佳常量回答分别约为 1/2、1/3、1/2，平均为 4/9≈44.4%。这比把七种答案混在一起说“随机率 1/7”更符合任务结构。","sha256":"25a797783c0d33246a29e6b05de2a88236b882aa0783b1729ebf8616970cba5d","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6c0a1352ca05871bfa05f262","nextBlock":"b-804742a6dd17730405f1c272","prerequisites":[{"id":"b-111ba4b937bba57f308b5cc8","url":"#b-111ba4b937bba57f308b5cc8","role":"section-title"},{"id":"b-bc13c340b943c002ea96aad1","url":"#b-bc13c340b943c002ea96aad1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-804742a6dd17730405f1c272","type":"pre","scope":"block-ef7e95916f594784b1269ebf22f89911","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-804742a6dd17730405f1c272","content":"```\ndef draw_scene(count,left_color,right_shape,rng):\n    img=Image.new('RGB',(40,32),(244,242,236));d=ImageDraw.Draw(img)\n    slots=[10] if count==1 else [10,30]\n    for j,cx in enumerate(slots):\n        cy=16+int(rng.integers(-3,4));r=int(rng.integers(4,7));cx+=int(rng.integers(-1,2))\n        color=left_color if j==0 else str(rng.choice(list(COLORS)))\n        shape=right_shape if j==len(slots)-1 else str(rng.choice(['square','circle']))\n        box=(cx-r,cy-r,cx+r,cy+r)\n        if shape=='square':d.rectangle(box,fill=COLORS[color])\n        else:d.ellipse(box,fill=COLORS[color])\n    arr=np.array(img,dtype=np.float32)/255\n    return np.clip(arr+rng.normal(0,.014,arr.shape),0,1).astype(np.float32)\n\ndef make_vqa(n,seed,biased):\n    rng=np.random.default_rng(seed);images=[];answers=[];meta=[]\n    # Training has a 75% red prior. Test covers all 12 condition combinations equally.\n    combos=[(c,col,sh) for c in [1,2] for col in COLORS for sh in ['square','circle']]\n    for i in range(n):\n        if biased:\n            count=int(rng.choice([1,2]));color=str(rng.choice(list(COLORS),p=[.75,.125,.125]));shape=str(rng.choice(['square','circle']))\n        else:count,color,shape=combos[i%len(combos)]\n        images.append(draw_scene(count,color,shape,rng))\n        answers.append([A['one' if count==1 else 'two'],A[color],A[shape]])\n        meta.append({'count':count,'left_color':color,'right_shape':shape})\n    return torch.tensor(np.stack(images)).permute(0,3,1,2),torch.tensor(answers),meta\nXtr,Ytr,train_meta=make_vqa(720,14031,True)\nXte,Yte,test_meta=make_vqa(180,14032,False)\ntrain_hashes={hashlib.sha256(x.numpy().tobytes()).hexdigest() for x in Xtr}\nassert not train_hashes.intersection(hashlib.sha256(x.numpy().tobytes()).hexdigest() for x in Xte)\n# Metadata remains only in this data/label construction and plot captions.\n# model.forward receives tensors (pixels, question_id), never these dictionaries.\nnp.savez_compressed(OUT/'lab14-data.npz',train_images=Xtr.numpy(),train_answers=Ytr.numpy(),test_images=Xte.numpy(),test_answers=Yte.numpy())\nprint('train/test image counts:',len(Xtr),len(Xte),'pairs:',Ytr.numel(),Yte.numel())\nprint('train left-color prior:',{c:sum(m['left_color']==c for m in train_meta)/len(train_meta) for c in COLORS})\nprint('test left-color prior:',{c:sum(m['left_color']==c for m in test_meta)/len(test_meta) for c in COLORS})\n```","sha256":"36c815acc00c365f774968586f380e1fed0fcacf2b8ed3f268abf3b2845d3299","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-20b57075949948ac8d6711cd4621f5b6","title":"2. 先按图像切分，避免同一图的不同问题泄漏","number":4},"sectionTitle":"2. 先按图像切分，避免同一图的不同问题泄漏","currentQuestion":"2. 先按图像切分，避免同一图的不同问题泄漏","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-784289b33e0961699486872d","nextBlock":null,"prerequisites":[{"id":"b-111ba4b937bba57f308b5cc8","url":"#b-111ba4b937bba57f308b5cc8","role":"section-title"},{"id":"b-bc13c340b943c002ea96aad1","url":"#b-bc13c340b943c002ea96aad1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-f973e05d78936921ede3bc6e","type":"h2","scope":"block-62876c0daff44e6f9b582bcaa0d0dd00","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-f973e05d78936921ede3bc6e","content":"## 3. 模型、损失与公平比较范围","sha256":"3129d617c9d64f12477cc585d4a415fb35853a79d91697ee3a67e21cf439afc7","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-63535d833b2eabff2b29b66d","prerequisites":[{"id":"b-63535d833b2eabff2b29b66d","url":"#b-63535d833b2eabff2b29b66d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-63535d833b2eabff2b29b66d","type":"p","scope":"block-62876c0daff44e6f9b582bcaa0d0dd00","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-63535d833b2eabff2b29b66d","content":"像素分支是两层 CNN 与保留布局的全连接表示；问题分支是 12 维可学习嵌入。拼接后输出七个答案的 logits，以交叉熵 $-\\log p(a\\mid I,q)$ 训练。question-only 使用相同问题嵌入维度、答案空间与训练样本，但省去 CNN。两者都真正更新参数，不能把手填“red”冒充训练的语言基线。","sha256":"bd1543535de10e14989b8f47346a883f97b200294db444f7b421779ebcfdcd67","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"-\\log p(a\\mid I,q)","definitionContext":"3. 模型、损失与公平比较范围"}],"previousBlock":"b-f973e05d78936921ede3bc6e","nextBlock":"b-8768a540b23aaca30ba96bf8","prerequisites":[{"id":"b-f973e05d78936921ede3bc6e","url":"#b-f973e05d78936921ede3bc6e","role":"section-title"}],"relatedBlocks":[]},{"id":"b-8768a540b23aaca30ba96bf8","type":"p","scope":"block-62876c0daff44e6f9b582bcaa0d0dd00","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-8768a540b23aaca30ba96bf8","content":"完整模型参数更多，因此二者差距本身不是严格的等参数容量结论。我们在同一个完整模型上又做像素置零与错配，固定参数再移除正确图像条件。该消融更直接检验本次预测是否使用图像。","sha256":"8eef598675a5a3470afd723119d407b25beb4b74308e2af413c81ebdca7c6581","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-63535d833b2eabff2b29b66d","nextBlock":"b-b4feadbbb801d4d4074bb762","prerequisites":[{"id":"b-f973e05d78936921ede3bc6e","url":"#b-f973e05d78936921ede3bc6e","role":"section-title"},{"id":"b-63535d833b2eabff2b29b66d","url":"#b-63535d833b2eabff2b29b66d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b4feadbbb801d4d4074bb762","type":"pre","scope":"block-7f07825ae8a84326b6a7a768b26c7caf","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-b4feadbbb801d4d4074bb762","content":"```\nclass VQA(nn.Module):\n    def __init__(self,visual):\n        super().__init__();self.visual=visual\n        self.question=nn.Embedding(len(QUESTIONS),12)\n        if visual:\n            self.cnn=nn.Sequential(nn.Conv2d(3,12,3,padding=1),nn.ReLU(),nn.MaxPool2d(2),nn.Conv2d(12,24,3,padding=1),nn.ReLU(),nn.MaxPool2d(2),nn.Flatten(),nn.Linear(24*8*10,64),nn.ReLU())\n        self.answer=nn.Sequential(nn.Linear(64+12 if visual else 12,64),nn.ReLU(),nn.Linear(64,len(ANSWERS)))\n    def forward(self,pixels,question_id):\n        q=self.question(question_id)\n        features=torch.cat([self.cnn(pixels),q],-1) if self.visual else q\n        return self.answer(features)\n\nfull=VQA(True);question_only=VQA(False)\ncurve=[]\nfor label,net in [('image+question',full),('question-only',question_only)]:\n    optimizer=torch.optim.Adam(net.parameters(),lr=.002)\n    for epoch in range(28):\n        order=torch.randperm(Ytr.numel());net.train();loss_sum=0.\n        for pair_ids in order.split(72):\n            image_ids=pair_ids//3;qids=pair_ids%3\n            logits=net(Xtr[image_ids],qids);target=Ytr[image_ids,qids]\n            loss=F.cross_entropy(logits,target)\n            optimizer.zero_grad();loss.backward();optimizer.step();loss_sum+=float(loss.detach())*len(pair_ids)\n        if epoch in [0,5,13,27]:print(label,'epoch',epoch+1,'CE',round(loss_sum/Ytr.numel(),4))\n        curve.append({'model':label,'epoch':epoch+1,'train_ce':loss_sum/Ytr.numel()})\n    torch.save(net.state_dict(),OUT/('lab14-'+('visual' if net.visual else 'question-only')+'-weights.pt'))\n```","sha256":"701155870a1e8d1c9076048a018c3eac278a5c5ce5d1530da54c7367d1727177","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-62876c0daff44e6f9b582bcaa0d0dd00","title":"3. 模型、损失与公平比较范围","number":5},"sectionTitle":"3. 模型、损失与公平比较范围","currentQuestion":"3. 模型、损失与公平比较范围","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8768a540b23aaca30ba96bf8","nextBlock":null,"prerequisites":[{"id":"b-f973e05d78936921ede3bc6e","url":"#b-f973e05d78936921ede3bc6e","role":"section-title"},{"id":"b-63535d833b2eabff2b29b66d","url":"#b-63535d833b2eabff2b29b66d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-100c1206964361ca121d899e","type":"h2","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-100c1206964361ca121d899e","content":"## 4. 逐问题读结果，别只看总准确率","sha256":"1fe68838bac1f3e94024769f31316795734a73b8163daa4afe9dffeb19a16c42","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-fbbc153f2d4f0c8e493fede2","prerequisites":[{"id":"b-4bccd162cb617dd51f18bb8f","url":"#b-4bccd162cb617dd51f18bb8f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-fbbc153f2d4f0c8e493fede2","type":"table","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-fbbc153f2d4f0c8e493fede2","content":"| 保留集条件 | 数量 | 左侧颜色 | 右侧形状 | 全部 |\n| --- | --- | --- | --- | --- |\n| Image + question | 100.0% | 100.0% | 100.0% | 100.0% |\n| Question-only | 50.0% | 33.3% | 50.0% | 44.4% |\n| 同一模型，像素置零 | 50.0% | 33.3% | 50.0% | 44.4% |\n| 同一模型，图像循环错配 | 83.3% | 50.0% | 0.0% | 44.4% |","sha256":"c973352dfecd3c1e7c8a26ed089a8951087a83f682f227b3de93c8dcc5168934","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["保留集条件","数量","左侧颜色","右侧形状","全部"],"rows":[["保留集条件","数量","左侧颜色","右侧形状","全部"],["Image + question","100.0%","100.0%","100.0%","100.0%"],["Question-only","50.0%","33.3%","50.0%","44.4%"],["同一模型，像素置零","50.0%","33.3%","50.0%","44.4%"],["同一模型，图像循环错配","83.3%","50.0%","0.0%","44.4%"]]},"previousBlock":"b-100c1206964361ca121d899e","nextBlock":"b-4bccd162cb617dd51f18bb8f","prerequisites":[{"id":"b-100c1206964361ca121d899e","url":"#b-100c1206964361ca121d899e","role":"section-title"},{"id":"b-4bccd162cb617dd51f18bb8f","url":"#b-4bccd162cb617dd51f18bb8f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-4bccd162cb617dd51f18bb8f","type":"p","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-4bccd162cb617dd51f18bb8f","content":"循环错配不是各属性独立重抽：数据按组合排列，所以相邻样本常保留数量，却总改变形状。于是 83.3%/50%/0% 不是异常；它正提醒我们干预协议必须说清。不要把这个排列解释为各任务的随机机会水平。","sha256":"5772517b1201f9efe13bb04cb40965793bbfb2044defb920a18a4690ca93b8af","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-fbbc153f2d4f0c8e493fede2","nextBlock":"b-f8a807e341bc9ea4085373d6","prerequisites":[{"id":"b-100c1206964361ca121d899e","url":"#b-100c1206964361ca121d899e","role":"section-title"}],"relatedBlocks":[]},{"id":"b-f8a807e341bc9ea4085373d6","type":"p","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-f8a807e341bc9ea4085373d6","content":"![相同保留问答上的视觉、语言、零图像与错配图像比较](/assets/display/fe08ead67458f0103fca897b.webp)","sha256":"96dafa969bed5d9ad470f33155f80bcacac14d744136bb6b734389f81e9e5cf9","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-4bccd162cb617dd51f18bb8f","nextBlock":"b-4b1e2c58cda67c89133dfc1f","prerequisites":[{"id":"b-100c1206964361ca121d899e","url":"#b-100c1206964361ca121d899e","role":"section-title"},{"id":"b-4bccd162cb617dd51f18bb8f","url":"#b-4bccd162cb617dd51f18bb8f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-4b1e2c58cda67c89133dfc1f","type":"p","scope":"block-4eb1163b8bc64dd5ab088d22e10318bb","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-4b1e2c58cda67c89133dfc1f","content":"同图换问展示问题控制“读什么”，同问换图展示视觉控制“答什么”。先查控制图中红/蓝只变颜色的一对，再看代码记录的九条问答，判断本次证据是否同时满足这两个方向。","sha256":"c51336f20e1d778413d93dfee9689f8717a4228ddc7f0a4593271786481d265b","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f8a807e341bc9ea4085373d6","nextBlock":"b-240e1c1b77fcc6c7c6898754","prerequisites":[{"id":"b-100c1206964361ca121d899e","url":"#b-100c1206964361ca121d899e","role":"section-title"},{"id":"b-4bccd162cb617dd51f18bb8f","url":"#b-4bccd162cb617dd51f18bb8f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-240e1c1b77fcc6c7c6898754","type":"pre","scope":"block-3ce456f62e2a4489be2b14a56d1b568e","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-240e1c1b77fcc6c7c6898754","content":"```\n@torch.no_grad()\ndef answer_all(net,images):\n    net.eval()\n    return torch.stack([net(images,torch.full((len(images),),q)).argmax(-1) for q in range(3)],1)\nnormal=answer_all(full,Xte);qo=answer_all(question_only,Xte)\nzero=answer_all(full,torch.zeros_like(Xte));shuffled=answer_all(full,Xte[torch.roll(torch.arange(len(Xte)),1)])\ntrain_full=answer_all(full,Xtr);train_qo=answer_all(question_only,Xtr)\ndef score(p,y):return {'overall':float(p.eq(y).float().mean()),**{QUESTIONS[q]:float(p[:,q].eq(y[:,q]).float().mean()) for q in range(3)}}\nmetrics={'test_image_question':score(normal,Yte),'test_question_only':score(qo,Yte),'test_zero_pixels':score(zero,Yte),'test_shuffled_pixels':score(shuffled,Yte),'train_image_question':score(train_full,Ytr),'train_question_only':score(train_qo,Ytr)}\nprint(json.dumps(metrics,indent=2))\n# Recreate two images with matched nuisance randomness, only left color differs.\ncontrol_specs=[(2,'red','square'),(2,'blue','square'),(1,'green','circle')]\ncontrols_x=torch.tensor(np.stack([draw_scene(*s,np.random.default_rng(444)) for s in control_specs])).permute(0,3,1,2)\ncontrols_pred=answer_all(full,controls_x);controls_qo=answer_all(question_only,controls_x)\ncontrols=[{'image_condition':list(s),'question':q,'image_question':ANSWERS[int(controls_pred[i,j])],'question_only':ANSWERS[int(controls_qo[i,j])]} for i,s in enumerate(control_specs) for j,q in enumerate(QUESTIONS)]\nprint('same image / changed question and same question / changed image:')\nfor row in controls:print(row)\n```","sha256":"8e18a29d7fa6b2a327c610746be654c9657a1db37fbbcdf87d48868eec182c2d","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-4eb1163b8bc64dd5ab088d22e10318bb","title":"4. 逐问题读结果，别只看总准确率","number":6},"sectionTitle":"4. 逐问题读结果，别只看总准确率","currentQuestion":"4. 逐问题读结果，别只看总准确率","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-4b1e2c58cda67c89133dfc1f","nextBlock":null,"prerequisites":[{"id":"b-100c1206964361ca121d899e","url":"#b-100c1206964361ca121d899e","role":"section-title"},{"id":"b-4bccd162cb617dd51f18bb8f","url":"#b-4bccd162cb617dd51f18bb8f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-66c28521a295f98b98be5efc","type":"h2","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-66c28521a295f98b98be5efc","content":"## 5. 学习曲线与可以被推翻的结论","sha256":"1ca0d2332e22d968ff90ecec13d4503cd081f51138359dfa0245b7402b49375f","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-26a733100939c30ca63aede5","prerequisites":[{"id":"b-26a733100939c30ca63aede5","url":"#b-26a733100939c30ca63aede5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-26a733100939c30ca63aede5","type":"p","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-26a733100939c30ca63aede5","content":"![图像问答与只读问题模型的实际训练交叉熵](/assets/display/d154401bc0457c19795d01aa.webp)","sha256":"ef326977eb3c2d61ef5d4269aeb7b71d502b9f00abd948c34e3a83fd1eff4bba","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-66c28521a295f98b98be5efc","nextBlock":"b-f34764783ef77c063ed37e11","prerequisites":[{"id":"b-66c28521a295f98b98be5efc","url":"#b-66c28521a295f98b98be5efc","role":"section-title"}],"relatedBlocks":[]},{"id":"b-f34764783ef77c063ed37e11","type":"p","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-f34764783ef77c063ed37e11","content":"完整模型在这个有限合成分布上达到 100%，不等于解决真实 VQA。颜色、边缘和对象数都容易辨别，问题模板也固定。只改一个条件：保持图像不变，把未知改写“which hue is on the left?”输入当前 ID 接口会发生什么？正确结论是它根本没有这个文本编码能力，不应把手工映射改写到 ID 的功能归功于网络。","sha256":"beea7d2078e665fc442333a2e15bb69694a025ec8f0b7eb6e2db5b2861eb3a42","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-26a733100939c30ca63aede5","nextBlock":"b-5d36271c2a1005b62ee8f380","prerequisites":[{"id":"b-66c28521a295f98b98be5efc","url":"#b-66c28521a295f98b98be5efc","role":"section-title"},{"id":"b-26a733100939c30ca63aede5","url":"#b-26a733100939c30ca63aede5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-5d36271c2a1005b62ee8f380","type":"p","scope":"block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-5d36271c2a1005b62ee8f380","content":"另一项可运行改动是把训练颜色先验改为均匀，保留模型与测试不变。先预测 question-only 的训练颜色分数会下降，而保留集颜色分数仍约为三分之一。模型可以减少训练与测试的先验落差，却仍需要看图才能逐例回答。","sha256":"0632888c3952b95db1c950e8a9167e33450ef30280bad500214018db5eedcf4b","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f34764783ef77c063ed37e11","nextBlock":"b-3a920f103a300341bebe33ce","prerequisites":[{"id":"b-66c28521a295f98b98be5efc","url":"#b-66c28521a295f98b98be5efc","role":"section-title"},{"id":"b-26a733100939c30ca63aede5","url":"#b-26a733100939c30ca63aede5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3a920f103a300341bebe33ce","type":"pre","scope":"block-040689f0e3594ff988d47a70c68dbab6","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-3a920f103a300341bebe33ce","content":"```\nfig,axes=plt.subplots(1,3,figsize=(11,4))\nfor i,ax in enumerate(axes):\n    ax.imshow(controls_x[i].permute(1,2,0));ax.axis('off')\n    rows=['count: ','left color: ','right shape: ']\n    text='\\n'.join(rows[j]+ANSWERS[int(controls_pred[i,j])]+' [Q-only: '+ANSWERS[int(controls_qo[i,j])]+']' for j in range(3))\n    ax.set_title(text,fontsize=10)\nfig.suptitle('LAB14: same questions, changed image; three questions per image',fontsize=13)\nfig.tight_layout();fig.savefig(OUT/'lab14-vqa-controls.png',dpi=160);plt.show();plt.close(fig)\nfig,ax=plt.subplots(figsize=(10,3.6));labels=['image + Q','Q only','zero pixels','shuffled pixels'];keys=['test_image_question','test_question_only','test_zero_pixels','test_shuffled_pixels'];values=[metrics[k]['overall'] for k in keys]\nax.bar(labels,values,color=['#356b91','#ad8251','#9da6ad','#9da6ad']);ax.set(ylim=(0,1.1),ylabel='answer accuracy (540 held-out pairs)')\nfor i,v in enumerate(values):ax.text(i,v+.02,f'{v:.1%}',ha='center')\nfig.tight_layout();fig.savefig(OUT/'lab14-vqa-ablation.png',dpi=160);plt.show();plt.close(fig)\nfig,ax=plt.subplots(figsize=(8,3.2))\nfor label in ['image+question','question-only']:\n    r=[v for v in curve if v['model']==label];ax.plot([v['epoch'] for v in r],[v['train_ce'] for v in r],label=label)\nax.set(xlabel='epoch',ylabel='answer cross entropy');ax.legend();fig.tight_layout();fig.savefig(OUT/'lab14-vqa-learning.png',dpi=160);plt.show();plt.close(fig)\nresult={'notebook_id':'11d9f74871544a77b42024a9fe06b81a','execution':'local CPU, fresh process; not a Deepnote cloud run','created_utc':datetime.now(timezone.utc).isoformat(),'elapsed_seconds':time.perf_counter()-START,'environment':{'python':sys.version,'torch':torch.__version__,'numpy':np.__version__,'platform':platform.platform()},'split':{'train_seed':14031,'test_seed':14032,'train_images':720,'test_images':180,'questions_per_image':3,'split_before_question_expansion':True,'train_red_prior':.75,'test_color_prior':'uniform by construction'},'metrics':metrics,'curve':curve,'controls':controls,'predictions':[{'truth':[ANSWERS[int(v)] for v in Yte[i]],'image_question':[ANSWERS[int(v)] for v in normal[i]],'question_only':[ANSWERS[int(v)] for v in qo[i]]} for i in range(len(Xte))],'files_sha256':{p.name:hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(OUT.glob('*.pt'))},'data_sha256':hashlib.sha256((OUT/'lab14-data.npz').read_bytes()).hexdigest()}\n(OUT/'lab14-results.json').write_text(json.dumps(result,indent=2));print('saved',OUT,'seconds',round(result['elapsed_seconds'],2))\n```","sha256":"2e78f928663ed19a4abe9bb865f2ec5b12eca912e29834ec3865f05c50ab3b0b","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-7bb1e0b0f13f4a8ca3f9aa6b7115edc8","title":"5. 学习曲线与可以被推翻的结论","number":7},"sectionTitle":"5. 学习曲线与可以被推翻的结论","currentQuestion":"5. 学习曲线与可以被推翻的结论","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-5d36271c2a1005b62ee8f380","nextBlock":null,"prerequisites":[{"id":"b-66c28521a295f98b98be5efc","url":"#b-66c28521a295f98b98be5efc","role":"section-title"},{"id":"b-26a733100939c30ca63aede5","url":"#b-26a733100939c30ca63aede5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-482d3ed9e12973d3e50394ac","type":"h2","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-482d3ed9e12973d3e50394ac","content":"## 结果文件、来源与回链","sha256":"8660b9343b7f1d52be6bcc92613183fa8e5476cd47e6bec2381d18175cddfcca","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-ed259a07be23e60a9cce758e","prerequisites":[{"id":"b-ed259a07be23e60a9cce758e","url":"#b-ed259a07be23e60a9cce758e","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ed259a07be23e60a9cce758e","type":"p","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-ed259a07be23e60a9cce758e","content":"`lab14-data.npz` 保存像素与答案；两个 `.pt` 文件对应两个真实训练模型；`lab14-results.json` 保存每类准确率、逐轮损失、九条受控问答、逐图保留预测及环境/数据哈希。场景元数据只用于监督标签与解读，未传入网络。","sha256":"be9dcfab95622299ac41f14246ffc693f6f04b7a84339f461ae3821b17c802ec","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-482d3ed9e12973d3e50394ac","nextBlock":"b-6934ca394df57a14baa917ee","prerequisites":[{"id":"b-482d3ed9e12973d3e50394ac","url":"#b-482d3ed9e12973d3e50394ac","role":"section-title"}],"relatedBlocks":[]},{"id":"b-6934ca394df57a14baa917ee","type":"p","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-6934ca394df57a14baa917ee","content":"[VQA 原论文，作者 arXiv 页面](https://arxiv.org/abs/1505.00468) 给出图像与问题到答案的任务定义与基线比较。[VQA 官方项目](https://visualqa.org/) 提供真实数据、评价与后续 VQA v2；本实验的精确分类准确率不是官方多人答案共识评分。我们的合成颜色先验是教学构造，不是该数据集的实测统计。","sha256":"605e525420770ca13650761ad0f9b0d9e751f8f5a21f8e29f246e4562f2f4a9e","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ed259a07be23e60a9cce758e","nextBlock":"b-19d3dd405f17209941df7d8f","prerequisites":[{"id":"b-482d3ed9e12973d3e50394ac","url":"#b-482d3ed9e12973d3e50394ac","role":"section-title"},{"id":"b-ed259a07be23e60a9cce758e","url":"#b-ed259a07be23e60a9cce758e","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-19d3dd405f17209941df7d8f","type":"p","scope":"block-193cc04bc38242b7be21838002bf9285","url":"https://codingai-lec03.pages.dev/course/lab14-视觉问答与语言捷径.html#b-19d3dd405f17209941df7d8f","content":"回到 [CLS 03](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/680da245145447dd88983aabd136fd24)：CLIP 排候选描述，caption 自回归生成，VQA 在问题条件下输出答案。它们共享视觉与语言，输出空间和训练目标却不同，不能因为界面都接受文字就互相替代。","sha256":"4bdd15d61276026eb047e031a5db6e0226050b63c5c87eeb2a31ae52cb226668","chapter":{"id":"course","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"unit":{"id":"lab14-视觉问答与语言捷径","title":"LAB 14 · 图像问答：视觉条件与问题先验"},"page":{"id":"reading-block-193cc04bc38242b7be21838002bf9285","title":"结果文件、来源与回链","number":8},"sectionTitle":"结果文件、来源与回链","currentQuestion":"结果文件、来源与回链","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6934ca394df57a14baa917ee","nextBlock":null,"prerequisites":[{"id":"b-482d3ed9e12973d3e50394ac","url":"#b-482d3ed9e12973d3e50394ac","role":"section-title"},{"id":"b-ed259a07be23e60a9cce758e","url":"#b-ed259a07be23e60a9cce758e","role":"problem-context"}],"relatedBlocks":[]}]}