{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"LAB 13 · 看图生成一句话：训练、解码与消融","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html","markdown":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.md","pages":[{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":1},{"id":"reading-block-b2fc4308b392433db454b46a8aea7e08","title":"先拆开两个容易混淆的问题","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":2},{"id":"reading-block-548cf71dafb04c7abed115b08c054a26","title":"1. 准备一个可以完整跑完的学习问题","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":3},{"id":"reading-block-b16f97bb031341f8b373532196bd0ca8","title":"2. 图像先生成，目标句子只进入损失","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":4},{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":5},{"id":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","title":"4. 推理只接自己的输出，再检查视觉依赖","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":6},{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":7},{"id":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","title":"保存、来源与下一步","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":8}],"aliases":[{"oldId":"b-35e52a526f62b6984e398c3c","targetId":"b-35e52a526f62b6984e398c3c","page":"reading-opening","operation":"move","beforeSHA256":"758ef1f636e48c8a25cff3ea44e0952bd1f125013acc4b9e4d3238483d4bcae5","afterSHA256":"758ef1f636e48c8a25cff3ea44e0952bd1f125013acc4b9e4d3238483d4bcae5","reason":"Same source body in authored semantic page"},{"oldId":"b-9e958e04140c96f836c886cb","targetId":"b-9e958e04140c96f836c886cb","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-dcb467974c14438182166764","targetId":"b-dcb467974c14438182166764","page":"reading-opening","operation":"move","beforeSHA256":"6a611aef342a7535362f318ee0c33b63170ce4ab853fa44a05f76a04873fd320","afterSHA256":"6a611aef342a7535362f318ee0c33b63170ce4ab853fa44a05f76a04873fd320","reason":"Same source body in authored semantic page"},{"oldId":"b-a07ca422009046e46eb10e4d","targetId":"b-a07ca422009046e46eb10e4d","page":"reading-opening","operation":"move","beforeSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","afterSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","reason":"Same source body in authored semantic page"},{"oldId":"b-7816f3e150f3945cbe2df5b4","targetId":"b-7816f3e150f3945cbe2df5b4","page":"reading-opening","operation":"move","beforeSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","afterSHA256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","reason":"Same source body in authored semantic page"},{"oldId":"b-87bd3ec4d42780bd91dbff0e","targetId":"b-87bd3ec4d42780bd91dbff0e","page":"reading-opening","operation":"move","beforeSHA256":"6d15dd7cd1e410c3b462ad12e5c096e99f52620e6ea410d32e1010982856d3fc","afterSHA256":"6d15dd7cd1e410c3b462ad12e5c096e99f52620e6ea410d32e1010982856d3fc","reason":"Same source body in authored semantic page"},{"oldId":"b-73e4845ebc85fc0b24486658","targetId":"b-73e4845ebc85fc0b24486658","page":"reading-opening","operation":"move","beforeSHA256":"936c1f791e72aa9829fba35a2f8a6cbed45e786138bccc4c8cbea1187ef81f38","afterSHA256":"936c1f791e72aa9829fba35a2f8a6cbed45e786138bccc4c8cbea1187ef81f38","reason":"Same source body in authored semantic page"},{"oldId":"b-a2c592ed2985ec0778e59b56","targetId":"b-a2c592ed2985ec0778e59b56","page":"reading-opening","operation":"move","beforeSHA256":"dbd8267617e65afb9b361c38ab5ecd9deb08703c519c39e9a9fa45ca552b9794","afterSHA256":"dbd8267617e65afb9b361c38ab5ecd9deb08703c519c39e9a9fa45ca552b9794","reason":"Same source body in authored semantic page"},{"oldId":"b-6413805d320f550933febc37","targetId":"b-6413805d320f550933febc37","page":"reading-opening","operation":"move","beforeSHA256":"890fbc23abc501b6a296dd283f5d91269fcdc5fe37222a92fa880db818085b66","afterSHA256":"f6aedf109a28f615cb59b69833b6929e30ff48023b1eabc37990766d7cfd0565","reason":"Same source body in authored semantic page"},{"oldId":"b-7d99a1b2766b04aed9c100a4","targetId":"b-7d99a1b2766b04aed9c100a4","page":"reading-opening","operation":"move","beforeSHA256":"1149558f23d8ace57289208a1034f60a2abcfdd08f4c6474343263d32efe6c1e","afterSHA256":"1149558f23d8ace57289208a1034f60a2abcfdd08f4c6474343263d32efe6c1e","reason":"Same source body in authored semantic page"},{"oldId":"b-6f79c8ef450745c8b3144f77","targetId":"b-6f79c8ef450745c8b3144f77","page":"reading-block-b2fc4308b392433db454b46a8aea7e08","operation":"move","beforeSHA256":"7d43b2cd612caac0d9ef9ed80d27ef50b9e15bf568494129532dab13d8e42dee","afterSHA256":"7d43b2cd612caac0d9ef9ed80d27ef50b9e15bf568494129532dab13d8e42dee","reason":"Same source body in authored semantic page"},{"oldId":"b-97cdda468a6169fedf0ef1e1","targetId":"b-97cdda468a6169fedf0ef1e1","page":"reading-block-b2fc4308b392433db454b46a8aea7e08","operation":"move","beforeSHA256":"474a42e855d5dbc7ef978a8f341d7491663d315e429a4e978d644f0d41f34db0","afterSHA256":"474a42e855d5dbc7ef978a8f341d7491663d315e429a4e978d644f0d41f34db0","reason":"Same source body in authored semantic page"},{"oldId":"b-8cdfb81f0709dec75b4beca1","targetId":"b-8cdfb81f0709dec75b4beca1","page":"reading-block-b2fc4308b392433db454b46a8aea7e08","operation":"move","beforeSHA256":"de4bb265855b5196b7fc032fcb9914a536028720debcb078fdffd6da83782416","afterSHA256":"de4bb265855b5196b7fc032fcb9914a536028720debcb078fdffd6da83782416","reason":"Same source body in authored semantic page"},{"oldId":"b-b0c2435f06ab3d2f997e812f","targetId":"b-b0c2435f06ab3d2f997e812f","page":"reading-block-b2fc4308b392433db454b46a8aea7e08","operation":"move","beforeSHA256":"fe612210a04db2d66535a4d2e55646467ca69911a367f10f2740b915aff04424","afterSHA256":"fe612210a04db2d66535a4d2e55646467ca69911a367f10f2740b915aff04424","reason":"Same source body in authored semantic page"},{"oldId":"b-da7c0322e7d85ef7298d2bee","targetId":"b-da7c0322e7d85ef7298d2bee","page":"reading-block-548cf71dafb04c7abed115b08c054a26","operation":"move","beforeSHA256":"94b9f53417513185203b2d0d918c12999ed0bd6eae815eb057d490cec4bffc4f","afterSHA256":"94b9f53417513185203b2d0d918c12999ed0bd6eae815eb057d490cec4bffc4f","reason":"Same source body in authored semantic page"},{"oldId":"b-6ad3c27d1ae31905146f9b75","targetId":"b-6ad3c27d1ae31905146f9b75","page":"reading-block-548cf71dafb04c7abed115b08c054a26","operation":"move","beforeSHA256":"dd9864336f3893d784aa8809e48a8a5b6a1a68d61aad093d049f251bccda5e41","afterSHA256":"dd9864336f3893d784aa8809e48a8a5b6a1a68d61aad093d049f251bccda5e41","reason":"Same source body in authored semantic page"},{"oldId":"b-638ae5d41c640a478d399b55","targetId":"b-638ae5d41c640a478d399b55","page":"reading-block-548cf71dafb04c7abed115b08c054a26","operation":"move","beforeSHA256":"01a48bc33bbbde9dae6971e46c8814469efae11d6cdad9b9e52cb5c925f30c70","afterSHA256":"01a48bc33bbbde9dae6971e46c8814469efae11d6cdad9b9e52cb5c925f30c70","reason":"Same source body in authored semantic page"},{"oldId":"b-c88247fef6fe35ad2c4dcd11","targetId":"b-c88247fef6fe35ad2c4dcd11","page":"reading-block-b16f97bb031341f8b373532196bd0ca8","operation":"move","beforeSHA256":"4f829f912abc11b646e03a0fbc6ce7f61caa5056a61acdb49db05cba51f115ca","afterSHA256":"4f829f912abc11b646e03a0fbc6ce7f61caa5056a61acdb49db05cba51f115ca","reason":"Same source body in authored semantic page"},{"oldId":"b-fcca6635f58c791f70cba2b5","targetId":"b-fcca6635f58c791f70cba2b5","page":"reading-block-b16f97bb031341f8b373532196bd0ca8","operation":"move","beforeSHA256":"c909f3e1c453edb79d820e46c390fb1a6a7835e4645270e8145e7d4c6193b5b2","afterSHA256":"c909f3e1c453edb79d820e46c390fb1a6a7835e4645270e8145e7d4c6193b5b2","reason":"Same source body in authored semantic page"},{"oldId":"b-945daad8487d030391890659","targetId":"b-945daad8487d030391890659","page":"reading-block-b16f97bb031341f8b373532196bd0ca8","operation":"move","beforeSHA256":"87b989e6d5719102c64af757ff5632425b87e5e6e34ba6f4565ea4a6c11b0b4c","afterSHA256":"87b989e6d5719102c64af757ff5632425b87e5e6e34ba6f4565ea4a6c11b0b4c","reason":"Same source body in authored semantic page"},{"oldId":"b-4e75c5cb225211ea32b44644","targetId":"b-4e75c5cb225211ea32b44644","page":"reading-block-b16f97bb031341f8b373532196bd0ca8","operation":"move","beforeSHA256":"b0288a9b8d5d88212522932be60299343d4938528cb8e7153f5a279ed971e4e0","afterSHA256":"b0288a9b8d5d88212522932be60299343d4938528cb8e7153f5a279ed971e4e0","reason":"Same source body in authored semantic page"},{"oldId":"b-45d6657fb92099ebb106824e","targetId":"b-45d6657fb92099ebb106824e","page":"reading-block-e7b880e8e242436992df27f02a3a2506","operation":"move","beforeSHA256":"84a8b4f5b7259e5f7eac7b28526d4cdb36c75dd3753a2f41bc5d449c22fa73dd","afterSHA256":"84a8b4f5b7259e5f7eac7b28526d4cdb36c75dd3753a2f41bc5d449c22fa73dd","reason":"Same source body in authored semantic page"},{"oldId":"b-3964d64969346f7aab1cfb02","targetId":"b-3964d64969346f7aab1cfb02","page":"reading-block-e7b880e8e242436992df27f02a3a2506","operation":"move","beforeSHA256":"5c9d510c1d7a3d21f74bf5a01052b1e1ee1bf8ef098c98e4cb1cc31e721b9e49","afterSHA256":"5c9d510c1d7a3d21f74bf5a01052b1e1ee1bf8ef098c98e4cb1cc31e721b9e49","reason":"Same source body in authored semantic page"},{"oldId":"b-96e9f4c1c6c56276a5dfc760","targetId":"b-96e9f4c1c6c56276a5dfc760","page":"reading-block-e7b880e8e242436992df27f02a3a2506","operation":"move","beforeSHA256":"c41542200dc86ff049ee954783114853bc6d71abbc3bbc3e4dfa4ec098bcce9e","afterSHA256":"c41542200dc86ff049ee954783114853bc6d71abbc3bbc3e4dfa4ec098bcce9e","reason":"Same source body in authored semantic page"},{"oldId":"b-c9d5ee5a6e5d6fd483117431","targetId":"b-c9d5ee5a6e5d6fd483117431","page":"reading-block-e7b880e8e242436992df27f02a3a2506","operation":"move","beforeSHA256":"a7a9d6c8a32680a641a952d4142bce66e88936bef853a8ee6e590ab5d5471792","afterSHA256":"a7a9d6c8a32680a641a952d4142bce66e88936bef853a8ee6e590ab5d5471792","reason":"Same source body in authored semantic page"},{"oldId":"b-23d04a12ca27b8178cf2441d","targetId":"b-23d04a12ca27b8178cf2441d","page":"reading-block-e7b880e8e242436992df27f02a3a2506","operation":"move","beforeSHA256":"1464747dc70ad5f323ac232fd0cebf01dd19e3c36331a819b7d66e25fae7bc5c","afterSHA256":"1464747dc70ad5f323ac232fd0cebf01dd19e3c36331a819b7d66e25fae7bc5c","reason":"Same source body in authored semantic page"},{"oldId":"b-7083ba6c262230731908d2ff","targetId":"b-7083ba6c262230731908d2ff","page":"reading-block-e7b880e8e242436992df27f02a3a2506","operation":"move","beforeSHA256":"43eae65db43f41518c687371f8a724225e24f260be6097247b06824ef2a230c2","afterSHA256":"43eae65db43f41518c687371f8a724225e24f260be6097247b06824ef2a230c2","reason":"Same source body in authored semantic page"},{"oldId":"b-2d882a2a2caed15676d1244d","targetId":"b-2d882a2a2caed15676d1244d","page":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","operation":"move","beforeSHA256":"f2d01d4cc65a2d6b8086b410e90681528f43d74afafdaecea97b1929b58cc8e9","afterSHA256":"f2d01d4cc65a2d6b8086b410e90681528f43d74afafdaecea97b1929b58cc8e9","reason":"Same source body in authored semantic page"},{"oldId":"b-c5ee648b1ca187794efce7bb","targetId":"b-c5ee648b1ca187794efce7bb","page":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","operation":"move","beforeSHA256":"afd24af6fb83a24b9e6b1c37d986650cb001aeabdb60796ba074ba78d03f830e","afterSHA256":"afd24af6fb83a24b9e6b1c37d986650cb001aeabdb60796ba074ba78d03f830e","reason":"Same source body in authored semantic page"},{"oldId":"b-a83cb2b46849e91840e66e8b","targetId":"b-a83cb2b46849e91840e66e8b","page":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","operation":"move","beforeSHA256":"16f7f71dd346b7d3c2ce1b02f0545c45b258a932b83c6536d6981d6d5af217c7","afterSHA256":"16f7f71dd346b7d3c2ce1b02f0545c45b258a932b83c6536d6981d6d5af217c7","reason":"Same source body in authored semantic page"},{"oldId":"b-dcf608d364a9748818029dd3","targetId":"b-dcf608d364a9748818029dd3","page":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","operation":"move","beforeSHA256":"7484037b201c9794c21fc86f5bd33a6f31f9cf5c2c99baf3cd95d5076e2a3da7","afterSHA256":"7484037b201c9794c21fc86f5bd33a6f31f9cf5c2c99baf3cd95d5076e2a3da7","reason":"Same source body in authored semantic page"},{"oldId":"b-7c31cfad17ece38368973e3b","targetId":"b-7c31cfad17ece38368973e3b","page":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","operation":"move","beforeSHA256":"a8795f74087fc20755c646a3e622120516eb1f0cf95407daf7336d82452e3833","afterSHA256":"a8795f74087fc20755c646a3e622120516eb1f0cf95407daf7336d82452e3833","reason":"Same source body in authored semantic page"},{"oldId":"b-989c9361d34480090478ecad","targetId":"b-989c9361d34480090478ecad","page":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","operation":"move","beforeSHA256":"449d8d044fcab3c7885899a1e56e732311d426f30e4a66f131a34bab08ae07aa","afterSHA256":"449d8d044fcab3c7885899a1e56e732311d426f30e4a66f131a34bab08ae07aa","reason":"Same source body in authored semantic page"},{"oldId":"b-b033f8b1c4dcf56d8b5c0887","targetId":"b-b033f8b1c4dcf56d8b5c0887","page":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","operation":"rewrite","beforeSHA256":"a5d7ef7ca31d88ce019b616c1fc80a89cea0e66ff297a98ab3b85533838822e0","afterSHA256":"3f2d373c284c1808045cd185627a8cb57ad6cd8a62e9ffb9b27287df448f052f","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-86d2753db6531d0983abaa60","targetId":"b-86d2753db6531d0983abaa60","page":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","operation":"move","beforeSHA256":"de649e205ea41f97c21e00dd76f96405b322f7331bb126d7177fbf3a99d0afb2","afterSHA256":"217c33a949219dc55fcf02f3b8197c19387428bbf8b30fd96d6a2d7874a426f4","reason":"Same source body in authored semantic page"},{"oldId":"b-cc9546be6cb6cd582506c301","targetId":"b-cc9546be6cb6cd582506c301","page":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","operation":"move","beforeSHA256":"acd69b54369171ecccd491fd1908af79010d8f9e1350d8ee6e8e5ce5b8ab29ca","afterSHA256":"06a62d71b18ea2894536a468a89044b5ec8c05501e03f28c8b88b24e951322bc","reason":"Same source body in authored semantic page"},{"oldId":"b-d0f6bb78869b9268159a526e","targetId":"b-d0f6bb78869b9268159a526e","page":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","operation":"move","beforeSHA256":"0e880ca0e640d4321b062e588bf4a7d632259a0789fa84443740eb518fce406d","afterSHA256":"0e880ca0e640d4321b062e588bf4a7d632259a0789fa84443740eb518fce406d","reason":"Same source body in authored semantic page"},{"oldId":"b-d8ac3c14c3b429c8a2aa1365","targetId":"b-d8ac3c14c3b429c8a2aa1365","page":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","operation":"move","beforeSHA256":"fe6b7b975ffb43f76795d7a6d1dd588c056cfb2c1266f38d7307a533a1177f11","afterSHA256":"fe6b7b975ffb43f76795d7a6d1dd588c056cfb2c1266f38d7307a533a1177f11","reason":"Same source body in authored semantic page"},{"oldId":"b-db46a78c7ecc373b73f71728","targetId":"b-db46a78c7ecc373b73f71728","page":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","operation":"move","beforeSHA256":"bd3982bbc4fd1e6afb787ccf52edb0e4b4285a14d74ad8c84919ee86cd4257b2","afterSHA256":"bd3982bbc4fd1e6afb787ccf52edb0e4b4285a14d74ad8c84919ee86cd4257b2","reason":"Same source body in authored semantic page"},{"oldId":"b-429caa8d652669e805b13044","targetId":"b-429caa8d652669e805b13044","page":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","operation":"move","beforeSHA256":"0854227e36b3988b7bf89699f125aa27774fbb64e41d56fc88439dbba2075d56","afterSHA256":"0854227e36b3988b7bf89699f125aa27774fbb64e41d56fc88439dbba2075d56","reason":"Same source body in authored semantic page"},{"oldId":"b-82bc7496e3e8388f881711ce","targetId":"b-82bc7496e3e8388f881711ce","page":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","operation":"move","beforeSHA256":"518fc2f5572e559ca862bf025b2c35e97789f71486330aa199438d0057ada022","afterSHA256":"518fc2f5572e559ca862bf025b2c35e97789f71486330aa199438d0057ada022","reason":"Same source body in authored semantic page"},{"oldId":"b-d25c5b195bc12bec1bee751d","targetId":"b-d25c5b195bc12bec1bee751d","page":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","operation":"move","beforeSHA256":"724a302f1409fdc2a9176811c9412f0608930ab1e70f9e0fd04219ae961a9c0f","afterSHA256":"724a302f1409fdc2a9176811c9412f0608930ab1e70f9e0fd04219ae961a9c0f","reason":"Same source body in authored semantic page"},{"oldId":"block-b2fc4308b392433db454b46a8aea7e08--先拆开两个容易混淆的问题","targetId":"b-6f79c8ef450745c8b3144f77","operation":"retain-source-identity"},{"oldId":"block-548cf71dafb04c7abed115b08c054a26--1-准备一个可以完整跑完的学习问题","targetId":"b-da7c0322e7d85ef7298d2bee","operation":"retain-source-identity"},{"oldId":"block-b16f97bb031341f8b373532196bd0ca8--2-图像先生成目标句子只进入损失","targetId":"b-c88247fef6fe35ad2c4dcd11","operation":"retain-source-identity"},{"oldId":"block-e7b880e8e242436992df27f02a3a2506--3-视觉特征怎样真正进入语言模型","targetId":"b-45d6657fb92099ebb106824e","operation":"retain-source-identity"},{"oldId":"block-fab7a942c9ab4a2ca8dc8d8c5d2877d9--4-推理只接自己的输出再检查视觉依赖","targetId":"b-2d882a2a2caed15676d1244d","operation":"retain-source-identity"},{"oldId":"block-16ef050e2f094a6eaf6b7c8f4f14dc32--5-读真实图真实曲线与失败边界","targetId":"b-989c9361d34480090478ecad","operation":"retain-source-identity"},{"oldId":"block-a41054827a0d49cdb9c0041dd9a7be4b--保存来源与下一步","targetId":"b-db46a78c7ecc373b73f71728","operation":"retain-source-identity"}],"blocks":[{"id":"b-35e52a526f62b6984e398c3c","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-35e52a526f62b6984e398c3c","content":"# LAB 13 · 看图生成一句话：训练、解码与消融","sha256":"758ef1f636e48c8a25cff3ea44e0952bd1f125013acc4b9e4d3238483d4bcae5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-9e958e04140c96f836c886cb","prerequisites":[{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9e958e04140c96f836c886cb","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-9e958e04140c96f836c886cb","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-35e52a526f62b6984e398c3c","nextBlock":"b-dcb467974c14438182166764","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"}],"relatedBlocks":[]},{"id":"b-dcb467974c14438182166764","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-dcb467974c14438182166764","content":"真实CNN–GRU条件训练与greedy/beam生成、遮图/错图消融","sha256":"6a611aef342a7535362f318ee0c33b63170ce4ab853fa44a05f76a04873fd320","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9e958e04140c96f836c886cb","nextBlock":"b-a07ca422009046e46eb10e4d","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a07ca422009046e46eb10e4d","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-a07ca422009046e46eb10e4d","content":"可先阅读已有结果，再决定是否运行。重算时请核对各节记录的数据、参数、环境与日期。","sha256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-dcb467974c14438182166764","nextBlock":"b-7816f3e150f3945cbe2df5b4","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7816f3e150f3945cbe2df5b4","type":"p","scope":"block-71a055b4f2af4411829bd08c5f697050","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-7816f3e150f3945cbe2df5b4","content":"**学习路径：** [图像怎样约束句子与回答](/course/study/caption-vqa.html)","sha256":"a57513e59eb07960710e5ece7951cc57af4a94c446630eb5fc6d70b460982042","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a07ca422009046e46eb10e4d","nextBlock":"b-87bd3ec4d42780bd91dbff0e","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-87bd3ec4d42780bd91dbff0e","type":"p","scope":"block-71a055b4f2af4411829bd08c5f697050","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-87bd3ec4d42780bd91dbff0e","content":"**同样以`BOS a`开头，换一张图能否生成不同句子？** 先读[P51](/course/pages/l3-r2-p51.html)与[逐词目标表](/course/blocks/caption-shift-and-stop.html)，再对照本册第2节输入/标签、第3节视觉进入路径、第4/5节实际生成和消融。","sha256":"6d15dd7cd1e410c3b462ad12e5c096e99f52620e6ea410d32e1010982856d3fc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7816f3e150f3945cbe2df5b4","nextBlock":"b-73e4845ebc85fc0b24486658","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-73e4845ebc85fc0b24486658","type":"p","scope":"block-71a055b4f2af4411829bd08c5f697050","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-73e4845ebc85fc0b24486658","content":"本册确实用RGB图像训练CNN＋GRU并接续自身输出。2026-09-22本地基准保存了正确图、像素置零和循环错图的结果；18种语义组合在训练/保留两侧都出现，检验的是新渲染样本，不是未见组合或自然照片泛化。[结果和完整协议](/course/blocks/caption-visual-controls.html)。","sha256":"936c1f791e72aa9829fba35a2f8a6cbed45e786138bccc4c8cbea1187ef81f38","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-87bd3ec4d42780bd91dbff0e","nextBlock":"b-a2c592ed2985ec0778e59b56","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-a2c592ed2985ec0778e59b56","type":"p","scope":"block-71a055b4f2af4411829bd08c5f697050","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-a2c592ed2985ec0778e59b56","content":"读完应能拿一组图片与生成句子说明视觉是否影响输出，以及为何teacher-forced交叉熵不能替代生成评价。动手复现时按原代码顺序执行；GRU内部与beam搜索可留后续课，不影响先理解图像条件。另一个输出接口见[LAB 14 问答](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/11d9f74871544a77b42024a9fe06b81a)。","sha256":"dbd8267617e65afb9b361c38ab5ecd9deb08703c519c39e9a9fa45ca552b9794","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-73e4845ebc85fc0b24486658","nextBlock":"b-6413805d320f550933febc37","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-6413805d320f550933febc37","type":"p","scope":"block-71a055b4f2af4411829bd08c5f697050","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-6413805d320f550933febc37","content":"![固定前缀下的真实生成](/assets/display/9ebe76d49a449b9940b6470f.webp)","sha256":"f6aedf109a28f615cb59b69833b6929e30ff48023b1eabc37990766d7cfd0565","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a2c592ed2985ec0778e59b56","nextBlock":"b-7d99a1b2766b04aed9c100a4","prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7d99a1b2766b04aed9c100a4","type":"p","scope":"block-71a055b4f2af4411829bd08c5f697050","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-7d99a1b2766b04aed9c100a4","content":"图中句子由已训练模型生成。下一项问题是“同图换问会怎样”，进入 [LAB 14 · 图像问答：视觉条件与问题先验](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/11d9f74871544a77b42024a9fe06b81a?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=11d9f74871544a77b42024a9fe06b81a&utm_term=get_notebook)。","sha256":"1149558f23d8ace57289208a1034f60a2abcfdd08f4c6474343263d32efe6c1e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-opening","title":"LAB 13 · 看图生成一句话：训练、解码与消融","number":1},"sectionTitle":"LAB 13 · 看图生成一句话：训练、解码与消融","currentQuestion":"LAB 13 · 看图生成一句话：训练、解码与消融","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6413805d320f550933febc37","nextBlock":null,"prerequisites":[{"id":"b-35e52a526f62b6984e398c3c","url":"#b-35e52a526f62b6984e398c3c","role":"section-title"},{"id":"b-9e958e04140c96f836c886cb","url":"#b-9e958e04140c96f836c886cb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-6f79c8ef450745c8b3144f77","type":"h2","scope":"block-b2fc4308b392433db454b46a8aea7e08","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-6f79c8ef450745c8b3144f77","content":"## 先拆开两个容易混淆的问题","sha256":"7d43b2cd612caac0d9ef9ed80d27ef50b9e15bf568494129532dab13d8e42dee","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b2fc4308b392433db454b46a8aea7e08","title":"先拆开两个容易混淆的问题","number":2},"sectionTitle":"先拆开两个容易混淆的问题","currentQuestion":"先拆开两个容易混淆的问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-97cdda468a6169fedf0ef1e1","prerequisites":[{"id":"b-97cdda468a6169fedf0ef1e1","url":"#b-97cdda468a6169fedf0ef1e1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-97cdda468a6169fedf0ef1e1","type":"p","scope":"block-b2fc4308b392433db454b46a8aea7e08","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-97cdda468a6169fedf0ef1e1","content":"训练的目标是 $p_\\theta(w_t\\mid I,w_{<t})$，不是同时猜出整句。teacher forcing 提供真实前缀，所以输入 `[BOS,a,red,square]` 对齐目标 `[a,red,square,EOS]`。PAD 只用来把批量补成矩形，不是下一词答案；EOS 则是真正需要学习的结束事件。下面保留原有两条整数序列，直接打印 input、target 与 mask。","sha256":"474a42e855d5dbc7ef978a8f341d7491663d315e429a4e978d644f0d41f34db0","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b2fc4308b392433db454b46a8aea7e08","title":"先拆开两个容易混淆的问题","number":2},"sectionTitle":"先拆开两个容易混淆的问题","currentQuestion":"先拆开两个容易混淆的问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"p_\\theta(w_t\\mid I,w_{<t})","definitionContext":"先拆开两个容易混淆的问题"}],"previousBlock":"b-6f79c8ef450745c8b3144f77","nextBlock":"b-8cdfb81f0709dec75b4beca1","prerequisites":[{"id":"b-6f79c8ef450745c8b3144f77","url":"#b-6f79c8ef450745c8b3144f77","role":"section-title"}],"relatedBlocks":[]},{"id":"b-8cdfb81f0709dec75b4beca1","type":"p","scope":"block-b2fc4308b392433db454b46a8aea7e08","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-8cdfb81f0709dec75b4beca1","content":"原概率树只写了 A→X 的 0.45，却把 AX 称为 greedy，没有交代另外 0.55 分给谁。现在补齐每层归一化的树：根的 A/B 为 0.55/0.45；A 的 X/Z 为 0.45/0.55；B 的 Y/Z 为 0.8/0.2。实际执行 greedy 得到 AZ，概率 0.3025；宽度 2 的 beam 得到 BY，概率 0.36。它解释局部最大不保证全句最大；并不证明 beam 在真实模型上一定改善内容正确性。","sha256":"de4bb265855b5196b7fc032fcb9914a536028720debcb078fdffd6da83782416","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b2fc4308b392433db454b46a8aea7e08","title":"先拆开两个容易混淆的问题","number":2},"sectionTitle":"先拆开两个容易混淆的问题","currentQuestion":"先拆开两个容易混淆的问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-97cdda468a6169fedf0ef1e1","nextBlock":"b-b0c2435f06ab3d2f997e812f","prerequisites":[{"id":"b-6f79c8ef450745c8b3144f77","url":"#b-6f79c8ef450745c8b3144f77","role":"section-title"},{"id":"b-97cdda468a6169fedf0ef1e1","url":"#b-97cdda468a6169fedf0ef1e1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b0c2435f06ab3d2f997e812f","type":"pre","scope":"block-3e80e4f0e33b470ba563baeb509e47c9","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-b0c2435f06ab3d2f997e812f","content":"```\n# Detached readonly runs write only to an isolated ephemeral directory.\nimport os, tempfile\n_LAB_WORKDIR = tempfile.mkdtemp(prefix=\"lecture03-lab13-\")\nos.chdir(_LAB_WORKDIR)\nprint(\"ephemeral workdir:\", _LAB_WORKDIR)\n\nseqs=[[1,4,5,2],[1,6,2,0]]\nprint('warm-up input',[s[:-1] for s in seqs])\nprint('warm-up target',[s[1:] for s in seqs])\nprint('warm-up loss mask',[[int(t!=0) for t in s[1:]] for s in seqs])\n# Every outgoing distribution sums to one. Both paths terminate after two tokens.\ntree={():{'A':.55,'B':.45},('A',):{'X':.45,'Z':.55},('B',):{'Y':.80,'Z':.20}}\nassert all(abs(sum(p.values())-1)<1e-9 for p in tree.values())\ngreedy_path=();greedy_probability=1.\nfor _ in range(2):\n    token=max(tree[greedy_path],key=tree[greedy_path].get)\n    greedy_probability*=tree[greedy_path][token];greedy_path+= (token,)\nbeams=[((),1.)]\nfor _ in range(2):\n    beams=sorted([(seq+(word,),p*conditional) for seq,p in beams for word,conditional in tree[seq].items()],key=lambda b:b[1],reverse=True)[:2]\nprint('actual tree greedy:',greedy_path,greedy_probability,'beam2:',beams[0])\n```","sha256":"fe612210a04db2d66535a4d2e55646467ca69911a367f10f2740b915aff04424","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b2fc4308b392433db454b46a8aea7e08","title":"先拆开两个容易混淆的问题","number":2},"sectionTitle":"先拆开两个容易混淆的问题","currentQuestion":"先拆开两个容易混淆的问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8cdfb81f0709dec75b4beca1","nextBlock":null,"prerequisites":[{"id":"b-6f79c8ef450745c8b3144f77","url":"#b-6f79c8ef450745c8b3144f77","role":"section-title"},{"id":"b-97cdda468a6169fedf0ef1e1","url":"#b-97cdda468a6169fedf0ef1e1","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-da7c0322e7d85ef7298d2bee","type":"h2","scope":"block-548cf71dafb04c7abed115b08c054a26","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-da7c0322e7d85ef7298d2bee","content":"## 1. 准备一个可以完整跑完的学习问题","sha256":"94b9f53417513185203b2d0d918c12999ed0bd6eae815eb057d490cec4bffc4f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-548cf71dafb04c7abed115b08c054a26","title":"1. 准备一个可以完整跑完的学习问题","number":3},"sectionTitle":"1. 准备一个可以完整跑完的学习问题","currentQuestion":"1. 准备一个可以完整跑完的学习问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-6ad3c27d1ae31905146f9b75","prerequisites":[{"id":"b-6ad3c27d1ae31905146f9b75","url":"#b-6ad3c27d1ae31905146f9b75","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-6ad3c27d1ae31905146f9b75","type":"p","scope":"block-548cf71dafb04c7abed115b08c054a26","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-6ad3c27d1ae31905146f9b75","content":"依赖 Python、NumPy、Pillow、Matplotlib、PyTorch；CPU 即可。缺包时在环境安装 `numpy pillow matplotlib torch`，随后 Restart & Run All。下面各格按顺序独立初始化数据与权重，不读另一册的变量，输出写入 `outputs/lab13/`。保存的基准使用 PyTorch 2.14.0+cpu；2026-09-22 保存基准的本地耗时约十余秒，不是 Deepnote 云端运行声明。","sha256":"dd9864336f3893d784aa8809e48a8a5b6a1a68d61aad093d049f251bccda5e41","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-548cf71dafb04c7abed115b08c054a26","title":"1. 准备一个可以完整跑完的学习问题","number":3},"sectionTitle":"1. 准备一个可以完整跑完的学习问题","currentQuestion":"1. 准备一个可以完整跑完的学习问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-da7c0322e7d85ef7298d2bee","nextBlock":"b-638ae5d41c640a478d399b55","prerequisites":[{"id":"b-da7c0322e7d85ef7298d2bee","url":"#b-da7c0322e7d85ef7298d2bee","role":"section-title"}],"relatedBlocks":[]},{"id":"b-638ae5d41c640a478d399b55","type":"pre","scope":"block-1543326fd7f749d382b10cc97d767713","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-638ae5d41c640a478d399b55","content":"```\nfrom pathlib import Path\nimport json, hashlib, platform, sys, time\nfrom datetime import datetime, timezone\nimport numpy as np\nfrom PIL import Image, ImageDraw\nimport matplotlib\nif 'get_ipython' not in globals(): matplotlib.use('Agg')\nimport matplotlib.pyplot as plt\nimport torch\nfrom torch import nn\nimport torch.nn.functional as F\nOUT = Path('outputs/lab13'); OUT.mkdir(parents=True, exist_ok=True)\ntorch.set_num_threads(1)\ntorch.manual_seed(1303)\nnp.random.seed(1303)\nSTART = time.perf_counter()\nVOCAB = ['<pad>', '<bos>', '<eos>', 'a', 'red', 'blue', 'green', 'square', 'circle', 'on', 'left', 'right']\nW = {s:i for i,s in enumerate(VOCAB)}\nPAD, BOS, EOS = 0, 1, 2\nCOLORS = {'red':(220,55,55), 'blue':(45,95,225), 'green':(35,170,75)}\nprint('CPU', torch.__version__, 'seed 1303; no downloaded data or weights')\n```","sha256":"01a48bc33bbbde9dae6971e46c8814469efae11d6cdad9b9e52cb5c925f30c70","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-548cf71dafb04c7abed115b08c054a26","title":"1. 准备一个可以完整跑完的学习问题","number":3},"sectionTitle":"1. 准备一个可以完整跑完的学习问题","currentQuestion":"1. 准备一个可以完整跑完的学习问题","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6ad3c27d1ae31905146f9b75","nextBlock":null,"prerequisites":[{"id":"b-da7c0322e7d85ef7298d2bee","url":"#b-da7c0322e7d85ef7298d2bee","role":"section-title"},{"id":"b-6ad3c27d1ae31905146f9b75","url":"#b-6ad3c27d1ae31905146f9b75","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c88247fef6fe35ad2c4dcd11","type":"h2","scope":"block-b16f97bb031341f8b373532196bd0ca8","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-c88247fef6fe35ad2c4dcd11","content":"## 2. 图像先生成，目标句子只进入损失","sha256":"4f829f912abc11b646e03a0fbc6ce7f61caa5056a61acdb49db05cba51f115ca","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b16f97bb031341f8b373532196bd0ca8","title":"2. 图像先生成，目标句子只进入损失","number":4},"sectionTitle":"2. 图像先生成，目标句子只进入损失","currentQuestion":"2. 图像先生成，目标句子只进入损失","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-fcca6635f58c791f70cba2b5","prerequisites":[{"id":"b-fcca6635f58c791f70cba2b5","url":"#b-fcca6635f58c791f70cba2b5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-fcca6635f58c791f70cba2b5","type":"p","scope":"block-b16f97bb031341f8b373532196bd0ca8","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-fcca6635f58c791f70cba2b5","content":"三种颜色 × 两种形状 × 三个位置，得到 18 种组合。每图随机改变小幅平移、尺寸和像素噪声。中心图像的描述较短，如 `a red square`；侧边图像则为 `a red square on left`，因此本实验真的存在 PAD，而不是摆一个从来不被使用的 mask。","sha256":"c909f3e1c453edb79d820e46c390fb1a6a7835e4645270e8145e7d4c6193b5b2","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b16f97bb031341f8b373532196bd0ca8","title":"2. 图像先生成，目标句子只进入损失","number":4},"sectionTitle":"2. 图像先生成，目标句子只进入损失","currentQuestion":"2. 图像先生成，目标句子只进入损失","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c88247fef6fe35ad2c4dcd11","nextBlock":"b-945daad8487d030391890659","prerequisites":[{"id":"b-c88247fef6fe35ad2c4dcd11","url":"#b-c88247fef6fe35ad2c4dcd11","role":"section-title"}],"relatedBlocks":[]},{"id":"b-945daad8487d030391890659","type":"p","scope":"block-b16f97bb031341f8b373532196bd0ca8","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-945daad8487d030391890659","content":"训练与保留图分别由 seed 13031 和 13033 生成，并检查像素哈希不重复。语义组合在两边都出现；这是新渲染样本的泛化，不是没见过组合的组合泛化。所有保留图的统计只用于最终报告。初期结构调试用 seed 13032 的开发图，另存于历史记录，不能混称独立测试。","sha256":"87b989e6d5719102c64af757ff5632425b87e5e6e34ba6f4565ea4a6c11b0b4c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b16f97bb031341f8b373532196bd0ca8","title":"2. 图像先生成，目标句子只进入损失","number":4},"sectionTitle":"2. 图像先生成，目标句子只进入损失","currentQuestion":"2. 图像先生成，目标句子只进入损失","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-fcca6635f58c791f70cba2b5","nextBlock":"b-4e75c5cb225211ea32b44644","prerequisites":[{"id":"b-c88247fef6fe35ad2c4dcd11","url":"#b-c88247fef6fe35ad2c4dcd11","role":"section-title"},{"id":"b-fcca6635f58c791f70cba2b5","url":"#b-fcca6635f58c791f70cba2b5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-4e75c5cb225211ea32b44644","type":"pre","scope":"block-61b73cd52382467c8d10d1143808ad17","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-4e75c5cb225211ea32b44644","content":"```\ndef render_object(color, shape, position, rng):\n    canvas = Image.new('RGB', (32,32), (244,242,236))\n    painter = ImageDraw.Draw(canvas)\n    cx = {'left':8, 'center':16, 'right':24}[position] + int(rng.integers(-1,2))\n    cy = 16 + int(rng.integers(-4,5)); radius = int(rng.integers(4,7))\n    box = (cx-radius, cy-radius, cx+radius, cy+radius)\n    if shape == 'square': painter.rectangle(box, fill=COLORS[color])\n    else: painter.ellipse(box, fill=COLORS[color])\n    arr = np.array(canvas).astype(np.float32) / 255\n    arr = np.clip(arr + rng.normal(0,.018,arr.shape),0,1).astype(np.float32)\n    return arr\n\ndef make_caption_data(n, seed):\n    rng = np.random.default_rng(seed); images=[]; sequences=[]; descriptions=[]\n    conditions = [(c,s,p) for c in COLORS for s in ['square','circle'] for p in ['left','center','right']]\n    for i in range(n):\n        color, shape, pos = conditions[i % len(conditions)]\n        words = ['a',color,shape] + ([] if pos=='center' else ['on',pos])\n        sequence = [BOS] + [W[w] for w in words] + [EOS]\n        sequences.append(sequence+[PAD]*(7-len(sequence)))\n        images.append(render_object(color,shape,pos,rng)); descriptions.append(' '.join(words))\n    return torch.tensor(np.stack(images)).permute(0,3,1,2), torch.tensor(sequences), descriptions\n\nXtrain, Ytrain, train_text = make_caption_data(720,13031)\nXtest, Ytest, test_text = make_caption_data(180,13033)\n# Independent random pixels/geometry; no duplicate image crosses the split.\ntrain_hashes = {hashlib.sha256(x.numpy().tobytes()).hexdigest() for x in Xtrain}\nassert not train_hashes.intersection(hashlib.sha256(x.numpy().tobytes()).hexdigest() for x in Xtest)\nnp.savez_compressed(OUT/'lab13-data.npz', train_images=Xtrain.numpy(), train_tokens=Ytrain.numpy(), test_images=Xtest.numpy(), test_tokens=Ytest.numpy())\nprint('images:', tuple(Xtrain.shape), tuple(Xtest.shape), 'tokens:', tuple(Ytrain.shape))\nprint('teacher input:', Ytrain[:3,:-1].tolist())\nprint('shifted target:',Ytrain[:3,1:].tolist())\nprint('loss mask:',Ytrain[:3,1:].ne(PAD).int().tolist())\n```","sha256":"b0288a9b8d5d88212522932be60299343d4938528cb8e7153f5a279ed971e4e0","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-b16f97bb031341f8b373532196bd0ca8","title":"2. 图像先生成，目标句子只进入损失","number":4},"sectionTitle":"2. 图像先生成，目标句子只进入损失","currentQuestion":"2. 图像先生成，目标句子只进入损失","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-945daad8487d030391890659","nextBlock":null,"prerequisites":[{"id":"b-c88247fef6fe35ad2c4dcd11","url":"#b-c88247fef6fe35ad2c4dcd11","role":"section-title"},{"id":"b-fcca6635f58c791f70cba2b5","url":"#b-fcca6635f58c791f70cba2b5","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-45d6657fb92099ebb106824e","type":"h2","scope":"block-e7b880e8e242436992df27f02a3a2506","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-45d6657fb92099ebb106824e","content":"## 3. 视觉特征怎样真正进入语言模型","sha256":"84a8b4f5b7259e5f7eac7b28526d4cdb36c75dd3753a2f41bc5d449c22fa73dd","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","number":5},"sectionTitle":"3. 视觉特征怎样真正进入语言模型","currentQuestion":"3. 视觉特征怎样真正进入语言模型","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-3964d64969346f7aab1cfb02","prerequisites":[{"id":"b-3964d64969346f7aab1cfb02","url":"#b-3964d64969346f7aab1cfb02","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3964d64969346f7aab1cfb02","type":"p","scope":"block-e7b880e8e242436992df27f02a3a2506","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-3964d64969346f7aab1cfb02","content":"输入是 `[B,3,32,32]`。两次卷积和池化产生空间特征，展平后映射为 48 维视觉向量；这里不用全局平均池化，因为“左/右”依赖空间布局。每个词嵌入为 24 维，与同一视觉向量拼接后送入共享的 GRU；视觉向量还初始化 64 维隐藏状态。输出 logits 为 `[B,T,12]`。","sha256":"5c9d510c1d7a3d21f74bf5a01052b1e1ee1bf8ef098c98e4cb1cc31e721b9e49","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","number":5},"sectionTitle":"3. 视觉特征怎样真正进入语言模型","currentQuestion":"3. 视觉特征怎样真正进入语言模型","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-45d6657fb92099ebb106824e","nextBlock":"b-96e9f4c1c6c56276a5dfc760","prerequisites":[{"id":"b-45d6657fb92099ebb106824e","url":"#b-45d6657fb92099ebb106824e","role":"section-title"}],"relatedBlocks":[]},{"id":"b-96e9f4c1c6c56276a5dfc760","type":"p","scope":"block-e7b880e8e242436992df27f02a3a2506","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-96e9f4c1c6c56276a5dfc760","content":"$L=-\\frac{\\sum_{i,t}\\mathbf 1[y_{it}\\ne PAD]\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})}{\\sum_{i,t}\\mathbf 1[y_{it}\\ne PAD]}.$","sha256":"c41542200dc86ff049ee954783114853bc6d71abbc3bbc3e4dfa4ec098bcce9e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","number":5},"sectionTitle":"3. 视觉特征怎样真正进入语言模型","currentQuestion":"3. 视觉特征怎样真正进入语言模型","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"L=-\\frac{\\sum_{i,t}\\mathbf 1[y_{it}\\ne PAD]\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})}{\\sum_{i,t}\\mathbf 1[y_{it}\\ne PAD]}.","definitionContext":"3. 视觉特征怎样真正进入语言模型"}],"previousBlock":"b-3964d64969346f7aab1cfb02","nextBlock":"b-c9d5ee5a6e5d6fd483117431","prerequisites":[{"id":"b-45d6657fb92099ebb106824e","url":"#b-45d6657fb92099ebb106824e","role":"section-title"},{"id":"b-3964d64969346f7aab1cfb02","url":"#b-3964d64969346f7aab1cfb02","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c9d5ee5a6e5d6fd483117431","type":"p","scope":"block-e7b880e8e242436992df27f02a3a2506","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-c9d5ee5a6e5d6fd483117431","content":"代码中的 `targets=Y[:,1:]` 对齐 `prefix=Y[:,:-1]`，`ignore_index=PAD` 排除补齐位置。优化器同时更新 CNN、词嵌入、GRU 和输出头，没有把真实颜色或位置喂给 CNN。每个 epoch 的保留集交叉熵仍是 teacher-forced 量，因此不能替代下一节的自由生成评价。","sha256":"a7a9d6c8a32680a641a952d4142bce66e88936bef853a8ee6e590ab5d5471792","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","number":5},"sectionTitle":"3. 视觉特征怎样真正进入语言模型","currentQuestion":"3. 视觉特征怎样真正进入语言模型","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-96e9f4c1c6c56276a5dfc760","nextBlock":"b-23d04a12ca27b8178cf2441d","prerequisites":[{"id":"b-45d6657fb92099ebb106824e","url":"#b-45d6657fb92099ebb106824e","role":"section-title"},{"id":"b-3964d64969346f7aab1cfb02","url":"#b-3964d64969346f7aab1cfb02","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-23d04a12ca27b8178cf2441d","type":"p","scope":"block-e7b880e8e242436992df27f02a3a2506","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-23d04a12ca27b8178cf2441d","content":"**一次真实失败也有教学意义：** 首轮使用未中心化像素和 tanh 视觉投影时，模型几乎忽略图像，整句准确率为 5.56%，视觉置零结果相同；历史代码、权重和 JSON 均保留。最终结构改为输入中心化、LayerNorm/ReLU 投影并降低学习率，之后锁定新的保留集。结果支持“视觉依赖需要检验”，不支持只凭训练损失下降就宣布模型看图。","sha256":"1464747dc70ad5f323ac232fd0cebf01dd19e3c36331a819b7d66e25fae7bc5c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","number":5},"sectionTitle":"3. 视觉特征怎样真正进入语言模型","currentQuestion":"3. 视觉特征怎样真正进入语言模型","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c9d5ee5a6e5d6fd483117431","nextBlock":"b-7083ba6c262230731908d2ff","prerequisites":[{"id":"b-45d6657fb92099ebb106824e","url":"#b-45d6657fb92099ebb106824e","role":"section-title"},{"id":"b-3964d64969346f7aab1cfb02","url":"#b-3964d64969346f7aab1cfb02","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7083ba6c262230731908d2ff","type":"pre","scope":"block-7ac2f2deac214e4a8db54c94134ff009","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-7083ba6c262230731908d2ff","content":"```\nclass Captioner(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.cnn = nn.Sequential(nn.Conv2d(3,12,3,padding=1),nn.ReLU(),nn.MaxPool2d(2),nn.Conv2d(12,24,3,padding=1),nn.ReLU(),nn.MaxPool2d(2),nn.Flatten(),nn.Linear(24*8*8,48),nn.LayerNorm(48),nn.ReLU())\n        self.word = nn.Embedding(len(VOCAB),24,padding_idx=PAD)\n        self.initial = nn.Linear(48,64)\n        self.rnn = nn.GRU(24+48,64,batch_first=True)\n        self.output = nn.Linear(64,len(VOCAB))\n    def encode(self, pixels):\n        return self.cnn(pixels - 0.5)\n    def forward(self, pixels, prefix):\n        visual = self.encode(pixels)\n        features = visual[:,None,:].expand(-1,prefix.shape[1],-1)\n        hidden,_ = self.rnn(torch.cat((self.word(prefix),features),-1),self.initial(visual)[None])\n        return self.output(hidden)\n\nmodel = Captioner()\noptimizer = torch.optim.Adam(model.parameters(),lr=.001)\ncurve=[]\nfor epoch in range(45):\n    model.train(); order=torch.randperm(len(Xtrain)); total=0.; tokens=0\n    for ix in order.split(60):\n        logits=model(Xtrain[ix],Ytrain[ix,:-1])\n        targets=Ytrain[ix,1:]\n        # Sum over real targets then divide by their count, not padded positions.\n        loss=F.cross_entropy(logits.reshape(-1,len(VOCAB)),targets.reshape(-1),ignore_index=PAD,reduction='sum')/targets.ne(PAD).sum()\n        optimizer.zero_grad(); loss.backward(); nn.utils.clip_grad_norm_(model.parameters(),5); optimizer.step()\n        count=int(targets.ne(PAD).sum()); total+=float(loss.detach())*count; tokens+=count\n    model.eval()\n    with torch.no_grad():\n        logits=model(Xtest,Ytest[:,:-1]); gold=Ytest[:,1:]\n        test_loss=float(F.cross_entropy(logits.reshape(-1,len(VOCAB)),gold.reshape(-1),ignore_index=PAD))\n    curve.append({'epoch':epoch+1,'train_token_ce':total/tokens,'heldout_teacher_forced_ce':test_loss})\n    if epoch in [0,5,11,23,44]: print(curve[-1])\ntorch.save({'state_dict':model.state_dict(),'vocab':VOCAB,'seed':1303},OUT/'lab13-caption-weights.pt')\n```","sha256":"43eae65db43f41518c687371f8a724225e24f260be6097247b06824ef2a230c2","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-e7b880e8e242436992df27f02a3a2506","title":"3. 视觉特征怎样真正进入语言模型","number":5},"sectionTitle":"3. 视觉特征怎样真正进入语言模型","currentQuestion":"3. 视觉特征怎样真正进入语言模型","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-23d04a12ca27b8178cf2441d","nextBlock":null,"prerequisites":[{"id":"b-45d6657fb92099ebb106824e","url":"#b-45d6657fb92099ebb106824e","role":"section-title"},{"id":"b-3964d64969346f7aab1cfb02","url":"#b-3964d64969346f7aab1cfb02","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2d882a2a2caed15676d1244d","type":"h2","scope":"block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-2d882a2a2caed15676d1244d","content":"## 4. 推理只接自己的输出，再检查视觉依赖","sha256":"f2d01d4cc65a2d6b8086b410e90681528f43d74afafdaecea97b1929b58cc8e9","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","title":"4. 推理只接自己的输出，再检查视觉依赖","number":6},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-c5ee648b1ca187794efce7bb","prerequisites":[{"id":"b-c5ee648b1ca187794efce7bb","url":"#b-c5ee648b1ca187794efce7bb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c5ee648b1ca187794efce7bb","type":"p","scope":"block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-c5ee648b1ca187794efce7bb","content":"Greedy 每一步取当前分布的 argmax，追加到自己的前缀，遇到 EOS 停止。下面的 beam 真实展开每条未结束假设，累加 log probability，并只保留分数最高的 3 条；结束假设不再扩展，最长生成 8 个新 token。此处没有长度归一化，所以不同长度会受原始句子概率的长度偏好影响。保留数据已足够简单，两种搜索都能答对，并未显示 beam 胜出。","sha256":"afd24af6fb83a24b9e6b1c37d986650cb001aeabdb60796ba074ba78d03f830e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","title":"4. 推理只接自己的输出，再检查视觉依赖","number":6},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2d882a2a2caed15676d1244d","nextBlock":"b-a83cb2b46849e91840e66e8b","prerequisites":[{"id":"b-2d882a2a2caed15676d1244d","url":"#b-2d882a2a2caed15676d1244d","role":"section-title"}],"relatedBlocks":[]},{"id":"b-a83cb2b46849e91840e66e8b","type":"p","scope":"block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-a83cb2b46849e91840e66e8b","content":"本册的整句精确匹配比较解码后的内容字符串：颜色、形状和位置需与规范句一致。`words()`会忽略BOS/PAD并在EOS处截断，因此这个分数不单独检查EOS出现的具体位置；停止行为应另看生成token序列。它适合唯一规范句子的合成任务。自然图像可有多种正确描述，不能照搬此指标。视觉置零是分布外干预，单独下降还不够；因此再做固定错配：把保留集图像循环移位一个位置，保持真值句子不变。这个排列无固定点，但不等于均匀随机抽样，也不能当作“随机猜测基线”。","sha256":"16f7f71dd346b7d3c2ce1b02f0545c45b258a932b83c6536d6981d6d5af217c7","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","title":"4. 推理只接自己的输出，再检查视觉依赖","number":6},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c5ee648b1ca187794efce7bb","nextBlock":"b-dcf608d364a9748818029dd3","prerequisites":[{"id":"b-2d882a2a2caed15676d1244d","url":"#b-2d882a2a2caed15676d1244d","role":"section-title"},{"id":"b-c5ee648b1ca187794efce7bb","url":"#b-c5ee648b1ca187794efce7bb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-dcf608d364a9748818029dd3","type":"p","scope":"block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-dcf608d364a9748818029dd3","content":"先预测：固定 `<bos> a`，红方块左侧换成蓝圆右侧，哪个词最先应改变？随后检查 `same_prefix_controls`，不要把数据生成器的标签当成模型生成。","sha256":"7484037b201c9794c21fc86f5bd33a6f31f9cf5c2c99baf3cd95d5076e2a3da7","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","title":"4. 推理只接自己的输出，再检查视觉依赖","number":6},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a83cb2b46849e91840e66e8b","nextBlock":"b-7c31cfad17ece38368973e3b","prerequisites":[{"id":"b-2d882a2a2caed15676d1244d","url":"#b-2d882a2a2caed15676d1244d","role":"section-title"},{"id":"b-c5ee648b1ca187794efce7bb","url":"#b-c5ee648b1ca187794efce7bb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7c31cfad17ece38368973e3b","type":"pre","scope":"block-c21718fad7a840a380c87175f7067cf5","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-7c31cfad17ece38368973e3b","content":"```\n@torch.no_grad()\ndef greedy(pixels, prefix=(BOS,), max_new=8):\n    model.eval(); seq=torch.tensor(prefix).repeat(len(pixels),1); finished=torch.zeros(len(pixels),dtype=torch.bool)\n    for _ in range(max_new):\n        nxt=model(pixels,seq)[:,-1].argmax(-1)\n        nxt=torch.where(finished,torch.tensor(EOS),nxt)\n        seq=torch.cat((seq,nxt[:,None]),1); finished|=nxt.eq(EOS)\n        if bool(finished.all()): break\n    return seq.tolist()\n\n@torch.no_grad()\ndef beam_one(pixel, width=3, max_new=8):\n    beams=[([BOS],0.0,False)]\n    for _ in range(max_new):\n        candidates=[]\n        for seq,score,done in beams:\n            if done: candidates.append((seq,score,True)); continue\n            lp=F.log_softmax(model(pixel[None],torch.tensor([seq]))[0,-1],dim=-1)\n            values,indices=lp.topk(width)\n            for value,idx in zip(values.tolist(),indices.tolist()):\n                candidates.append((seq+[idx],score+value,idx==EOS))\n        beams=sorted(candidates,key=lambda b:b[1],reverse=True)[:width]\n        if all(b[2] for b in beams): break\n    # Ended hypotheses only if at least one ended; otherwise max-length fallback.\n    ended=[b for b in beams if b[2]]\n    return max(ended or beams,key=lambda b:b[1])\n\ndef words(ids):\n    out=[]\n    for idx in ids:\n        if idx==EOS: break\n        if idx not in [BOS,PAD]: out.append(VOCAB[idx])\n    return ' '.join(out)\n\npred=[words(s) for s in greedy(Xtest)]\nbeam_pred=[words(beam_one(x)[0]) for x in Xtest]\nzero_pred=[words(s) for s in greedy(torch.zeros_like(Xtest))]\n# A deterministic derangement shifts test images; target captions stay fixed.\nshuffle_index=torch.roll(torch.arange(len(Xtest)),1)\nshuffle_pred=[words(s) for s in greedy(Xtest[shuffle_index])]\nexact=lambda p: float(np.mean([a==b for a,b in zip(p,test_text)]))\nmetrics={'heldout_images':len(Xtest),'greedy_exact_match':exact(pred),'beam3_exact_match':exact(beam_pred),'zero_pixels_exact_match':exact(zero_pred),'shuffled_pixels_exact_match':exact(shuffle_pred),'final_train_token_ce':curve[-1]['train_token_ce'],'final_heldout_teacher_forced_ce':curve[-1]['heldout_teacher_forced_ce']}\nprint(json.dumps(metrics,indent=2))\n# Hold the language prefix fixed; change only the input image.\nrng=np.random.default_rng(999)\ncontrol_specs=[('red','square','left'),('blue','circle','right'),('green','square','center')]\ncontrol_pixels=torch.tensor(np.stack([render_object(*spec,rng) for spec in control_specs])).permute(0,3,1,2)\ncontrol_pred=[words(s) for s in greedy(control_pixels,prefix=(BOS,W['a']))]\ncontrols=[{'condition':list(spec),'fixed_prefix':'<bos> a','generated':p} for spec,p in zip(control_specs,control_pred)]\nprint('same prefix, changed image:',json.dumps(controls,indent=2))\n```","sha256":"a8795f74087fc20755c646a3e622120516eb1f0cf95407daf7336d82452e3833","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-fab7a942c9ab4a2ca8dc8d8c5d2877d9","title":"4. 推理只接自己的输出，再检查视觉依赖","number":6},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-dcf608d364a9748818029dd3","nextBlock":null,"prerequisites":[{"id":"b-2d882a2a2caed15676d1244d","url":"#b-2d882a2a2caed15676d1244d","role":"section-title"},{"id":"b-c5ee648b1ca187794efce7bb","url":"#b-c5ee648b1ca187794efce7bb","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-989c9361d34480090478ecad","type":"h2","scope":"block-16ef050e2f094a6eaf6b7c8f4f14dc32","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-989c9361d34480090478ecad","content":"## 5. 读真实图、真实曲线与失败边界","sha256":"449d8d044fcab3c7885899a1e56e732311d426f30e4a66f131a34bab08ae07aa","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","number":7},"sectionTitle":"5. 读真实图、真实曲线与失败边界","currentQuestion":"5. 读真实图、真实曲线与失败边界","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-b033f8b1c4dcf56d8b5c0887","prerequisites":[{"id":"b-86d2753db6531d0983abaa60","url":"#b-86d2753db6531d0983abaa60","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b033f8b1c4dcf56d8b5c0887","type":"table","scope":"block-16ef050e2f094a6eaf6b7c8f4f14dc32","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-b033f8b1c4dcf56d8b5c0887","content":"| 解码/干预 | 保留集整句准确率 |\n| --- | --- |\n| Greedy，自回归 | 100.0% |\n| Beam，宽度 3 | 100.0% |\n| 相同模型，所有像素置零 | 5.6% |\n| 相同模型，图像循环错配 | 0.0% |","sha256":"3f2d373c284c1808045cd185627a8cb57ad6cd8a62e9ffb9b27287df448f052f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","number":7},"sectionTitle":"5. 读真实图、真实曲线与失败边界","currentQuestion":"5. 读真实图、真实曲线与失败边界","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["解码/干预","保留集整句准确率"],"rows":[["解码/干预","保留集整句准确率"],["Greedy，自回归","100.0%"],["Beam，宽度 3","100.0%"],["相同模型，所有像素置零","5.6%"],["相同模型，图像循环错配","0.0%"]]},"previousBlock":"b-989c9361d34480090478ecad","nextBlock":"b-86d2753db6531d0983abaa60","prerequisites":[{"id":"b-989c9361d34480090478ecad","url":"#b-989c9361d34480090478ecad","role":"section-title"},{"id":"b-86d2753db6531d0983abaa60","url":"#b-86d2753db6531d0983abaa60","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-86d2753db6531d0983abaa60","type":"p","scope":"block-16ef050e2f094a6eaf6b7c8f4f14dc32","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-86d2753db6531d0983abaa60","content":"![保留图像、真值描述及greedy和beam真实生成句子](/assets/display/a9bc909f343835d4ec673d45.webp)","sha256":"217c33a949219dc55fcf02f3b8197c19387428bbf8b30fd96d6a2d7874a426f4","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","number":7},"sectionTitle":"5. 读真实图、真实曲线与失败边界","currentQuestion":"5. 读真实图、真实曲线与失败边界","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b033f8b1c4dcf56d8b5c0887","nextBlock":"b-cc9546be6cb6cd582506c301","prerequisites":[{"id":"b-989c9361d34480090478ecad","url":"#b-989c9361d34480090478ecad","role":"section-title"}],"relatedBlocks":[]},{"id":"b-cc9546be6cb6cd582506c301","type":"p","scope":"block-16ef050e2f094a6eaf6b7c8f4f14dc32","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-cc9546be6cb6cd582506c301","content":"![训练和保留逐词交叉熵及图像干预后的整句准确率](/assets/display/b3d5136b8b5c8bfe245797e9.webp)","sha256":"06a62d71b18ea2894536a468a89044b5ec8c05501e03f28c8b88b24e951322bc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","number":7},"sectionTitle":"5. 读真实图、真实曲线与失败边界","currentQuestion":"5. 读真实图、真实曲线与失败边界","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-86d2753db6531d0983abaa60","nextBlock":"b-d0f6bb78869b9268159a526e","prerequisites":[{"id":"b-989c9361d34480090478ecad","url":"#b-989c9361d34480090478ecad","role":"section-title"},{"id":"b-86d2753db6531d0983abaa60","url":"#b-86d2753db6531d0983abaa60","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d0f6bb78869b9268159a526e","type":"p","scope":"block-16ef050e2f094a6eaf6b7c8f4f14dc32","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-d0f6bb78869b9268159a526e","content":"100% 只覆盖小词表、单对象和熟悉的语义组合；大小变化、噪声幅度都有限。下一步只改变一个条件：把 `radius` 范围扩大，或让图像包含两个对象，并先明确描述目标是否也应改变。不要在同一次改动中同时加对象、换语料和换评分，否则失败难以归因。","sha256":"0e880ca0e640d4321b062e588bf4a7d632259a0789fa84443740eb518fce406d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","number":7},"sectionTitle":"5. 读真实图、真实曲线与失败边界","currentQuestion":"5. 读真实图、真实曲线与失败边界","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-cc9546be6cb6cd582506c301","nextBlock":"b-d8ac3c14c3b429c8a2aa1365","prerequisites":[{"id":"b-989c9361d34480090478ecad","url":"#b-989c9361d34480090478ecad","role":"section-title"},{"id":"b-86d2753db6531d0983abaa60","url":"#b-86d2753db6531d0983abaa60","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d8ac3c14c3b429c8a2aa1365","type":"pre","scope":"block-1a6a0f13eab24338b41badd5b17c9678","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-d8ac3c14c3b429c8a2aa1365","content":"```\nfig,axes=plt.subplots(2,3,figsize=(10,6))\nchosen=[0,3,7,10,14,17]\nfor ax,i in zip(axes.flat,chosen):\n    ax.imshow(Xtest[i].permute(1,2,0)); ax.axis('off')\n    ax.set_title('truth: '+test_text[i]+'\\ngreedy: '+pred[i]+'\\nbeam: '+beam_pred[i],fontsize=9)\nfig.suptitle('LAB13: held-out images; model-generated captions',fontsize=13); fig.tight_layout()\nfig.savefig(OUT/'lab13-caption-predictions.png',dpi=160); plt.show(); plt.close(fig)\nfig,axes=plt.subplots(1,2,figsize=(10,3.4))\naxes[0].plot([r['epoch'] for r in curve],[r['train_token_ce'] for r in curve],label='train'); axes[0].plot([r['epoch'] for r in curve],[r['heldout_teacher_forced_ce'] for r in curve],label='held-out'); axes[0].set(xlabel='epoch',ylabel='mean non-pad token CE');axes[0].legend()\nlabels=['greedy','beam 3','zero image','shuffled image'];values=[metrics[k] for k in ['greedy_exact_match','beam3_exact_match','zero_pixels_exact_match','shuffled_pixels_exact_match']]\naxes[1].bar(labels,values,color=['#356b91','#40868e','#a9adb0','#a9adb0']);axes[1].set(ylim=(0,1.12),ylabel='exact caption accuracy (180 images)')\nfor i,v in enumerate(values):axes[1].text(i,v+.025,f'{v:.1%}',ha='center')\nfig.tight_layout();fig.savefig(OUT/'lab13-caption-controls.png',dpi=160);plt.show();plt.close(fig)\nfig,axes=plt.subplots(1,3,figsize=(9,3))\nfor ax,x,row in zip(axes,control_pixels,controls):\n    ax.imshow(x.permute(1,2,0));ax.axis('off');ax.set_title('prefix: <bos> a\\n'+row['generated'],fontsize=10)\nfig.tight_layout();fig.savefig(OUT/'lab13-same-prefix.png',dpi=160);plt.show();plt.close(fig)\nresult={'notebook_id':'3da47f3ee3554ee0a36e0bb6a2fdc61f','execution':'local CPU, fresh process; not a Deepnote cloud run','created_utc':datetime.now(timezone.utc).isoformat(),'elapsed_seconds':time.perf_counter()-START,'environment':{'python':sys.version,'torch':torch.__version__,'numpy':np.__version__,'platform':platform.platform()},'split':{'train_seed':13031,'test_seed':13033,'train_images':720,'test_images':180,'description':'independent noisy renders; same 18 semantic combinations on both sides; no compositional/general-photo claim'},'metrics':metrics,'curve':curve,'same_prefix_controls':controls,'predictions':[{'truth':t,'greedy':p,'beam3':b,'zero':z,'shuffled':s} for t,p,b,z,s in zip(test_text,pred,beam_pred,zero_pred,shuffle_pred)],'weight_sha256':hashlib.sha256((OUT/'lab13-caption-weights.pt').read_bytes()).hexdigest(),'data_sha256':hashlib.sha256((OUT/'lab13-data.npz').read_bytes()).hexdigest()}\n(OUT/'lab13-results.json').write_text(json.dumps(result,indent=2))\nprint('saved',OUT,'seconds',round(result['elapsed_seconds'],2))\n```","sha256":"fe6b7b975ffb43f76795d7a6d1dd588c056cfb2c1266f38d7307a533a1177f11","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-16ef050e2f094a6eaf6b7c8f4f14dc32","title":"5. 读真实图、真实曲线与失败边界","number":7},"sectionTitle":"5. 读真实图、真实曲线与失败边界","currentQuestion":"5. 读真实图、真实曲线与失败边界","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d0f6bb78869b9268159a526e","nextBlock":null,"prerequisites":[{"id":"b-989c9361d34480090478ecad","url":"#b-989c9361d34480090478ecad","role":"section-title"},{"id":"b-86d2753db6531d0983abaa60","url":"#b-86d2753db6531d0983abaa60","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-db46a78c7ecc373b73f71728","type":"h2","scope":"block-a41054827a0d49cdb9c0041dd9a7be4b","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-db46a78c7ecc373b73f71728","content":"## 保存、来源与下一步","sha256":"bd3982bbc4fd1e6afb787ccf52edb0e4b4285a14d74ad8c84919ee86cd4257b2","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","title":"保存、来源与下一步","number":8},"sectionTitle":"保存、来源与下一步","currentQuestion":"保存、来源与下一步","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-429caa8d652669e805b13044","prerequisites":[{"id":"b-429caa8d652669e805b13044","url":"#b-429caa8d652669e805b13044","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-429caa8d652669e805b13044","type":"p","scope":"block-a41054827a0d49cdb9c0041dd9a7be4b","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-429caa8d652669e805b13044","content":"`lab13-data.npz` 保存真实训练/保留像素与 token；`lab13-caption-weights.pt` 保存该次模型；`lab13-results.json` 保存分割种子、逐轮损失、每条预测、环境、时间和权重/数据 SHA-256。下载的执行版 Notebook 含实际输出，完整代码在每格内。本册的训练和推理已在本地全新 Python 进程运行；这与原 Deepnote 云端历史 run 分开记录。","sha256":"0854227e36b3988b7bf89699f125aa27774fbb64e41d56fc88439dbba2075d56","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","title":"保存、来源与下一步","number":8},"sectionTitle":"保存、来源与下一步","currentQuestion":"保存、来源与下一步","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-db46a78c7ecc373b73f71728","nextBlock":"b-82bc7496e3e8388f881711ce","prerequisites":[{"id":"b-db46a78c7ecc373b73f71728","url":"#b-db46a78c7ecc373b73f71728","role":"section-title"}],"relatedBlocks":[]},{"id":"b-82bc7496e3e8388f881711ce","type":"p","scope":"block-a41054827a0d49cdb9c0041dd9a7be4b","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-82bc7496e3e8388f881711ce","content":"[Show and Tell，CVPR 2015，论文作者/CVF 原文](https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Vinyals_Show_and_Tell_2015_CVPR_paper.html) 是图像条件最大似然描述生成的来源。本册保留这一因子分解与 CNN→循环解码思路，但使用小 GRU、自造图形和简化评价；不是原论文复现。","sha256":"518fc2f5572e559ca862bf025b2c35e97789f71486330aa199438d0057ada022","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","title":"保存、来源与下一步","number":8},"sectionTitle":"保存、来源与下一步","currentQuestion":"保存、来源与下一步","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-429caa8d652669e805b13044","nextBlock":"b-d25c5b195bc12bec1bee751d","prerequisites":[{"id":"b-db46a78c7ecc373b73f71728","url":"#b-db46a78c7ecc373b73f71728","role":"section-title"},{"id":"b-429caa8d652669e805b13044","url":"#b-429caa8d652669e805b13044","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d25c5b195bc12bec1bee751d","type":"p","scope":"block-a41054827a0d49cdb9c0041dd9a7be4b","url":"https://codingai-lec03.pages.dev/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html#b-d25c5b195bc12bec1bee751d","content":"回到 [第四讲 · 从条件多解走向生成模型](https://deepnote.com/project/6f83a923-d155-47be-be58-680db701ff7f/notebook/694b7a43ea4948f5a197c014c2fbad3b?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=694b7a43ea4948f5a197c014c2fbad3b&utm_term=get_notebook)：caption 已经在生成。下一讲继续研究的是怎样表达更复杂的条件分布、目标与采样，而不是第一次出现生成。","sha256":"724a302f1409fdc2a9176811c9412f0608930ab1e70f9e0fd04219ae961a9c0f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"reading-block-a41054827a0d49cdb9c0041dd9a7be4b","title":"保存、来源与下一步","number":8},"sectionTitle":"保存、来源与下一步","currentQuestion":"保存、来源与下一步","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-82bc7496e3e8388f881711ce","nextBlock":null,"prerequisites":[{"id":"b-db46a78c7ecc373b73f71728","url":"#b-db46a78c7ecc373b73f71728","role":"section-title"},{"id":"b-429caa8d652669e805b13044","url":"#b-429caa8d652669e805b13044","role":"problem-context"}],"relatedBlocks":[]}]}