{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"图像描述：视觉内容怎样成为一句话？","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html","markdown":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.md","pages":[{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":1},{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":2},{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","unit":"图像怎样约束一句新生成的描述？","unitId":"caption-visual-conditioning","chapter":"III","number":3},{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","unit":"一个条件怎样容纳多种合理结果？","unitId":"generation-many-possible-results","chapter":"III","number":4}],"aliases":[{"oldId":"explanation","targetId":"caption-image-to-next-token","operation":"split-explanation-range"},{"oldId":"b-52a55290d17cd6c64c30055c","targetId":"b-52a55290d17cd6c64c30055c","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","afterSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","reason":"Same source body in authored semantic page"},{"oldId":"b-ab410b4ca38fd3d426936f98","targetId":"b-ab410b4ca38fd3d426936f98","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"359603be813b7396cfc15cc3615d0203347bd58cab819c36c48b0642a668f877","afterSHA256":"359603be813b7396cfc15cc3615d0203347bd58cab819c36c48b0642a668f877","reason":"Same source body in authored semantic page"},{"oldId":"b-2cbe2468e7a524119d2c5266","targetId":"b-2cbe2468e7a524119d2c5266","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","afterSHA256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","reason":"Same source body in authored semantic page"},{"oldId":"b-14377e0b1fa48ceda8ad44c3","targetId":"b-14377e0b1fa48ceda8ad44c3","page":"caption-image-to-next-token","operation":"rewrite","beforeSHA256":"f53c165abbd842c81bbcb708d578cc640bb9b0a71d6f2431199f29555ac688ca","afterSHA256":"0e06fc8b3e1e98979520c451896ceecb0cb0e7c112adbed72ab8d8e12cd6a2e5","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-9af35ea50d7361f697eb68e6","targetId":"b-9af35ea50d7361f697eb68e6","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","afterSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","reason":"Same source body in authored semantic page"},{"oldId":"b-4d9e593b51c79d23f6b015f9","targetId":"b-4d9e593b51c79d23f6b015f9","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"ad0b4788ea26b16e9d703df8dbbae2c66f4c88f9b18f35b738020ec6a87f043c","afterSHA256":"ad0b4788ea26b16e9d703df8dbbae2c66f4c88f9b18f35b738020ec6a87f043c","reason":"Same source body in authored semantic page"},{"oldId":"b-5b415a76297390ec24af06c3","targetId":"b-5b415a76297390ec24af06c3","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"2ab0f86797acfc0468ac83ce362620f38848977a36abd44ce780bd36873300f5","afterSHA256":"2ab0f86797acfc0468ac83ce362620f38848977a36abd44ce780bd36873300f5","reason":"Same source body in authored semantic page"},{"oldId":"b-d910beacbe22be8799463f90","targetId":"b-d910beacbe22be8799463f90","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"73eb3160c6f026832b06ee63a280f8b54b7256670cc0346425cd8cb92efbd6e1","afterSHA256":"73eb3160c6f026832b06ee63a280f8b54b7256670cc0346425cd8cb92efbd6e1","reason":"Same source body in authored semantic page"},{"oldId":"b-26e241fe0c2038f9fb90eb5e","targetId":"b-26e241fe0c2038f9fb90eb5e","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"963b8cb4593a865d67c3b515ce7a59eda4fde2d526bf1fab3eedc10e8f3bbe72","afterSHA256":"963b8cb4593a865d67c3b515ce7a59eda4fde2d526bf1fab3eedc10e8f3bbe72","reason":"Same source body in authored semantic page"},{"oldId":"b-2cc4348120b97b10fd16ffb8","targetId":"b-2cc4348120b97b10fd16ffb8","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"38ac8dae0e03cd1374618f0f96fa050817f6b4ffb826dbe1ca1835086656d5be","afterSHA256":"38ac8dae0e03cd1374618f0f96fa050817f6b4ffb826dbe1ca1835086656d5be","reason":"Same source body in authored semantic page"},{"oldId":"b-76cf5db89bafdbd550277f80","targetId":"b-76cf5db89bafdbd550277f80","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"0c037e1d30329f16596e9cd66a94534e7997988977ea286a9927664b3809a313","afterSHA256":"0c037e1d30329f16596e9cd66a94534e7997988977ea286a9927664b3809a313","reason":"Same source body in authored semantic page"},{"oldId":"b-878ff244e0d49b6a9e673c8f","targetId":"b-878ff244e0d49b6a9e673c8f","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"9f55c5cb4129e8e8b797be8286c4a119ea9d0bf7f9a8b67d2cf9b22501f57e96","afterSHA256":"9f55c5cb4129e8e8b797be8286c4a119ea9d0bf7f9a8b67d2cf9b22501f57e96","reason":"Same source body in authored semantic page"},{"oldId":"b-b8f7ec9042ba6e44ae10e77e","targetId":"b-b8f7ec9042ba6e44ae10e77e","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"d6b83489df8601fc692dca4e701aa80750a0218408fbf8ac355ca7a8adee4249","afterSHA256":"d6b83489df8601fc692dca4e701aa80750a0218408fbf8ac355ca7a8adee4249","reason":"Same source body in authored semantic page"},{"oldId":"b-cb4c7be5b7131744ae04a8e5","targetId":"b-cb4c7be5b7131744ae04a8e5","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"06390678b1b9671244bcf735621682f5edfd65ca00ca25fa2011bf82d5b235f4","afterSHA256":"06390678b1b9671244bcf735621682f5edfd65ca00ca25fa2011bf82d5b235f4","reason":"Same source body in authored semantic page"},{"oldId":"b-b7df37fc48c089cfe2569a8f","targetId":"b-b7df37fc48c089cfe2569a8f","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"994b2a08a8cadd1cc3df33daa7df50aba5d07c339e218a42d7cce67b43964fe6","afterSHA256":"994b2a08a8cadd1cc3df33daa7df50aba5d07c339e218a42d7cce67b43964fe6","reason":"Same source body in authored semantic page"},{"oldId":"b-871676744729abf7dec49f40","targetId":"b-871676744729abf7dec49f40","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"2f42fe05aebc54159c3c3af4649940c3c73dfaaf6d4bf90ac7168ec5a626ca08","afterSHA256":"2f42fe05aebc54159c3c3af4649940c3c73dfaaf6d4bf90ac7168ec5a626ca08","reason":"Same source body in authored semantic page"},{"oldId":"b-fc16bf4219289e5738993935","targetId":"b-fc16bf4219289e5738993935","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"30c75f370a12cfa3e7ddb387dab529922355c84b1acd04f3cc00407380f879e1","afterSHA256":"30c75f370a12cfa3e7ddb387dab529922355c84b1acd04f3cc00407380f879e1","reason":"Same source body in authored semantic page"},{"oldId":"b-6fef75ac33b658bc054dc759","targetId":"b-6fef75ac33b658bc054dc759","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"16bc21ee8d6fe340ac60513d6125f0f5190a6d07566276dfa218beedddc0d9b3","afterSHA256":"16bc21ee8d6fe340ac60513d6125f0f5190a6d07566276dfa218beedddc0d9b3","reason":"Same source body in authored semantic page"},{"oldId":"b-2cd73f242adcad72ad553ab2","targetId":"b-2cd73f242adcad72ad553ab2","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"adca7a5d4689903412db69da89606445c541264728d3c505a705f275d08fd7d7","afterSHA256":"adca7a5d4689903412db69da89606445c541264728d3c505a705f275d08fd7d7","reason":"Same source body in authored semantic page"},{"oldId":"b-f44ed46f57462b6fb5289094","targetId":"b-f44ed46f57462b6fb5289094","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"c48baab888f8b3eec0764bc35694f23349c16c01f8c12ab5fc3efe8a94c69740","afterSHA256":"c48baab888f8b3eec0764bc35694f23349c16c01f8c12ab5fc3efe8a94c69740","reason":"Same source body in authored semantic page"},{"oldId":"b-a5aba144db2c50186c498741","targetId":"b-a5aba144db2c50186c498741","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"89a322d64437b3595d77045707a818d41f648d725ac24e3f284a6d8f51707cc3","afterSHA256":"89a322d64437b3595d77045707a818d41f648d725ac24e3f284a6d8f51707cc3","reason":"Same source body in authored semantic page"},{"oldId":"b-21714ca5e3e3202ccab6c32a","targetId":"b-21714ca5e3e3202ccab6c32a","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"2c172d358ceceb1a46e47a24cc98a2615f53e2848b17d6551df0e471d9a3ac52","afterSHA256":"2c172d358ceceb1a46e47a24cc98a2615f53e2848b17d6551df0e471d9a3ac52","reason":"Same source body in authored semantic page"},{"oldId":"b-8e3f623d987cbe3593ecea83","targetId":"b-8e3f623d987cbe3593ecea83","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"8fab6c6d454d29102d7d5a31edd6b6b7cf7d07a3daef7ad1dd882032726099c9","afterSHA256":"8fab6c6d454d29102d7d5a31edd6b6b7cf7d07a3daef7ad1dd882032726099c9","reason":"Same source body in authored semantic page"},{"oldId":"b-3d96dbf8c82c917088121158","targetId":"b-3d96dbf8c82c917088121158","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"31dd0c4957c4b012df377041af0122fadd7ef43ecc33fb3ae9cf182150a4d60e","afterSHA256":"31dd0c4957c4b012df377041af0122fadd7ef43ecc33fb3ae9cf182150a4d60e","reason":"Same source body in authored semantic page"},{"oldId":"b-0244a6799a6e12d1411f8175","targetId":"b-0244a6799a6e12d1411f8175","page":"caption-image-to-next-token","operation":"move","beforeSHA256":"ca368380e67807da96d28a3d42bac8a7130788bfad138122839a7f170a2e9e72","afterSHA256":"ca368380e67807da96d28a3d42bac8a7130788bfad138122839a7f170a2e9e72","reason":"Same source body in authored semantic page"},{"oldId":"b-c586b2c85bf8e363de087cee","targetId":"b-c586b2c85bf8e363de087cee","page":"caption-targets-and-stopping","operation":"move","beforeSHA256":"8c2505555011d4c3abd71b059fb64a24aae2fe9370ea4bc01db532cb1c16fba9","afterSHA256":"8c2505555011d4c3abd71b059fb64a24aae2fe9370ea4bc01db532cb1c16fba9","reason":"Same source body in authored semantic page"},{"oldId":"b-e462382727de092875ec35b4","targetId":"b-e462382727de092875ec35b4","page":"caption-targets-and-stopping","operation":"move","beforeSHA256":"dd0cbe5a9f571aff92f5177d64ed0ecef2a16c1713f740eea8cb5312f7a9f650","afterSHA256":"dd0cbe5a9f571aff92f5177d64ed0ecef2a16c1713f740eea8cb5312f7a9f650","reason":"Same source body in authored semantic page"},{"oldId":"b-a8af80f9dfed62aabe85f456","targetId":"b-a8af80f9dfed62aabe85f456","page":"caption-targets-and-stopping","operation":"rewrite","beforeSHA256":"791f8a35e18efd7bbc9a959937b6e461f00d5dc05cf985f8a69f8fa52f66f71c","afterSHA256":"7cf82523725f8dc1dbfc45a269f6e423480bce0858c9bf86dcb2bc5f68cfb9ab","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-f018399dadfb59081776c116","targetId":"b-f018399dadfb59081776c116","page":"caption-targets-and-stopping","operation":"move","beforeSHA256":"43952b0c901e562451395edc1a8bdeef87074554a5ad6cc25045b1d31c451f16","afterSHA256":"43952b0c901e562451395edc1a8bdeef87074554a5ad6cc25045b1d31c451f16","reason":"Same source body in authored semantic page"},{"oldId":"b-977d04e10b68031b8aa7a8cc","targetId":"b-977d04e10b68031b8aa7a8cc","page":"caption-targets-and-stopping","operation":"move","beforeSHA256":"abec1e77f4ee30f312e664a3c45a315e4bfe246e526b1dd7bb578311d2ada770","afterSHA256":"abec1e77f4ee30f312e664a3c45a315e4bfe246e526b1dd7bb578311d2ada770","reason":"Same source body in authored semantic page"},{"oldId":"b-adba7b27272722ca00ece129","targetId":"b-adba7b27272722ca00ece129","page":"caption-targets-and-stopping","operation":"move","beforeSHA256":"37cf03728db1dab4711e27ff47809b2be238f681a3a44e01bcb88c5744536edc","afterSHA256":"37cf03728db1dab4711e27ff47809b2be238f681a3a44e01bcb88c5744536edc","reason":"Same source body in authored semantic page"},{"oldId":"b-2d17b377c21a89d013046fac","targetId":"b-2d17b377c21a89d013046fac","page":"caption-lab13-visual-controls","operation":"move","beforeSHA256":"8254cd6b23c829245d1f704a82870a58d61049cf00ad79ec44e348371e74dbf3","afterSHA256":"8254cd6b23c829245d1f704a82870a58d61049cf00ad79ec44e348371e74dbf3","reason":"Same source body in authored semantic page"},{"oldId":"b-42556ad4acb0bbe952b01cab","targetId":"b-42556ad4acb0bbe952b01cab","page":"caption-lab13-visual-controls","operation":"move","beforeSHA256":"c1e2bdbd6a16066dce6e56451311b7266e3a63bc810800f91e36da038f829364","afterSHA256":"c1e2bdbd6a16066dce6e56451311b7266e3a63bc810800f91e36da038f829364","reason":"Same source body in authored semantic page"},{"oldId":"b-1b5258cda22c24c30e82c5ea","targetId":"b-1b5258cda22c24c30e82c5ea","page":"caption-lab13-visual-controls","operation":"move","beforeSHA256":"d6e2fdeea4c473a4ad3b0b4a0a761eeafdcdf897215012754c0cfc5db9ebb042","afterSHA256":"d6e2fdeea4c473a4ad3b0b4a0a761eeafdcdf897215012754c0cfc5db9ebb042","reason":"Same source body in authored semantic page"},{"oldId":"b-84aad620e02cc718409a4995","targetId":"b-84aad620e02cc718409a4995","page":"caption-lab13-visual-controls","operation":"rewrite","beforeSHA256":"8c976ba19945b3c06c115239582117ede7196427a44028e02bdff62b158b6800","afterSHA256":"6198a4971317db245f284fd359e91844808af55181f829edcbe77f7009c63809","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-ff901967d6958ea8eb362fd6","targetId":"b-ff901967d6958ea8eb362fd6","page":"caption-lab13-visual-controls","operation":"move","beforeSHA256":"1f615531076a140376b37f33392e5543bedc80c8ddb1b9b6483dd6d7e4d5f06b","afterSHA256":"1f615531076a140376b37f33392e5543bedc80c8ddb1b9b6483dd6d7e4d5f06b","reason":"Same source body in authored semantic page"},{"oldId":"b-65dab675304107e641dde759","targetId":"b-65dab675304107e641dde759","page":"caption-lab13-visual-controls","operation":"move","beforeSHA256":"890fbc23abc501b6a296dd283f5d91269fcdc5fe37222a92fa880db818085b66","afterSHA256":"f6aedf109a28f615cb59b69833b6929e30ff48023b1eabc37990766d7cfd0565","reason":"Same source body in authored semantic page"},{"oldId":"b-15c07ba6586c133142f20a91","targetId":"b-15c07ba6586c133142f20a91","page":"caption-lab13-visual-controls","operation":"move","beforeSHA256":"38fc170e9722124fda4bcec90baa5dd1ee7e03550b153f899912a90c1fbb4ce0","afterSHA256":"38fc170e9722124fda4bcec90baa5dd1ee7e03550b153f899912a90c1fbb4ce0","reason":"Same source body in authored semantic page"},{"oldId":"b-318eba2140cce74164a6d0b1","targetId":"b-318eba2140cce74164a6d0b1","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"44317b8cd70503c8912fc895871b14162b2f4220d9fe401770b3bd9f948c602d","afterSHA256":"44317b8cd70503c8912fc895871b14162b2f4220d9fe401770b3bd9f948c602d","reason":"Same source body in authored semantic page"},{"oldId":"b-83a4df7935e400f89d3aa427","targetId":"b-83a4df7935e400f89d3aa427","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"c69ff8d382b78b38f116e433914d074c2acb001bb346dd6ea11428f503fb1e9b","afterSHA256":"c69ff8d382b78b38f116e433914d074c2acb001bb346dd6ea11428f503fb1e9b","reason":"Same source body in authored semantic page"},{"oldId":"b-b7dda3daa921ecbce135ae19","targetId":"b-b7dda3daa921ecbce135ae19","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"ca0781f38ff1958fb130064abb90ad8c2678b32c1241d77839edd7ae1ef6acf8","afterSHA256":"ca0781f38ff1958fb130064abb90ad8c2678b32c1241d77839edd7ae1ef6acf8","reason":"Same source body in authored semantic page"},{"oldId":"b-fca31d6d6facb0c582ccf434","targetId":"b-fca31d6d6facb0c582ccf434","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"2263210ab03d3652966597d08ee87e2d2abef3bd071157eb51a6ea67ca8520ea","afterSHA256":"2263210ab03d3652966597d08ee87e2d2abef3bd071157eb51a6ea67ca8520ea","reason":"Same source body in authored semantic page"},{"oldId":"b-ef4ba79b1b52a3072a01a9fc","targetId":"b-ef4ba79b1b52a3072a01a9fc","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"92f197c441dd9a60ae9246f7eb4d102d3d469d13a8a8de0407e0c9c9aa009e0d","afterSHA256":"92f197c441dd9a60ae9246f7eb4d102d3d469d13a8a8de0407e0c9c9aa009e0d","reason":"Same source body in authored semantic page"},{"oldId":"b-178f6763e2fc8c22435ad239","targetId":"b-178f6763e2fc8c22435ad239","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","afterSHA256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","reason":"Same source body in authored semantic page"},{"oldId":"b-022daeeec6a74aadc8c6f57f","targetId":"b-022daeeec6a74aadc8c6f57f","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","afterSHA256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","reason":"Same source body in authored semantic page"},{"oldId":"b-f703d0bd8fe547374bb378fd","targetId":"b-f703d0bd8fe547374bb378fd","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"6cb9ccaf588b2cc164fe529844e1e6102136545b9d2b6a2e359c3851f8b42743","afterSHA256":"6cb9ccaf588b2cc164fe529844e1e6102136545b9d2b6a2e359c3851f8b42743","reason":"Same source body in authored semantic page"},{"oldId":"b-250d923240fb206c3daebcac","targetId":"b-250d923240fb206c3daebcac","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"cb83be2919d657ca9b1de372dca7c1599a21baa73d45328e9c9fb394a115595d","afterSHA256":"cb83be2919d657ca9b1de372dca7c1599a21baa73d45328e9c9fb394a115595d","reason":"Same source body in authored semantic page"},{"oldId":"b-62c8e50d63ea377f16864ffe","targetId":"b-62c8e50d63ea377f16864ffe","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"2856d764d26947c7c46b97ec0b11e760bbf74cdb47c3c344d3b51526d93b0ac6","afterSHA256":"2856d764d26947c7c46b97ec0b11e760bbf74cdb47c3c344d3b51526d93b0ac6","reason":"Same source body in authored semantic page"},{"oldId":"b-c643aa7bef222c64a4fe6a3a","targetId":"b-c643aa7bef222c64a4fe6a3a","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"fc5702a73e802790c290d2df3ed9892ca3e2fdb119b5d282bd0d8437da0bd775","afterSHA256":"fc5702a73e802790c290d2df3ed9892ca3e2fdb119b5d282bd0d8437da0bd775","reason":"Same source body in authored semantic page"},{"oldId":"b-e7f0b4057cd5c2def8bd2c94","targetId":"b-e7f0b4057cd5c2def8bd2c94","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","afterSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","reason":"Same source body in authored semantic page"},{"oldId":"b-fd9592072f20916bf4a1c605","targetId":"b-fd9592072f20916bf4a1c605","page":"caption-search-probability-extension","operation":"move","beforeSHA256":"d8c8ccccbd0b755c873b8e65c73b09831006e5dfdefc15ab254e6a0b7b49a5e0","afterSHA256":"d8c8ccccbd0b755c873b8e65c73b09831006e5dfdefc15ab254e6a0b7b49a5e0","reason":"Same source body in authored semantic page"},{"oldId":"caption-image-to-next-token-title","targetId":"b-51bae90881857a26744dacfd","operation":"retain-source-identity"},{"oldId":"caption-targets-and-stopping-title","targetId":"b-d3ade6c5e1c5567a00bc21f6","operation":"retain-source-identity"},{"oldId":"caption-targets-and-stopping-bridge","targetId":"b-288866af8ebe58e22e4d516a","operation":"retain-source-identity"},{"oldId":"caption-lab13-visual-controls-title","targetId":"b-3275b7767665ca864c0be4bc","operation":"retain-source-identity"},{"oldId":"caption-lab13-visual-controls-bridge","targetId":"b-ac3bb16f7db137b3d2d65f15","operation":"retain-source-identity"},{"oldId":"caption-search-probability-extension-title","targetId":"b-39315a7e537d87cbbe0dfbcc","operation":"retain-source-identity"},{"oldId":"caption-search-probability-extension-bridge","targetId":"b-93a7174a2a98e127794dcf5d","operation":"retain-source-identity"}],"blocks":[{"id":"b-51bae90881857a26744dacfd","type":"h2","scope":"caption-image-to-next-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-51bae90881857a26744dacfd","content":"## 沿视觉条件找到下一词目标","sha256":"a2db254a73f0a8530d7d54ac0e079267d9262a8b76d94bc60a1e343a50f5dd24","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-52a55290d17cd6c64c30055c","prerequisites":[{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-52a55290d17cd6c64c30055c","type":"h2","scope":"explanation","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-52a55290d17cd6c64c30055c","content":"## 解释、算例与证据","sha256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-51bae90881857a26744dacfd","nextBlock":"b-ab410b4ca38fd3d426936f98","prerequisites":[{"id":"b-51bae90881857a26744dacfd","url":"#b-51bae90881857a26744dacfd","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-ab410b4ca38fd3d426936f98","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-ab410b4ca38fd3d426936f98","content":"# P51 · 图像描述：视觉内容怎样成为一句话？","sha256":"359603be813b7396cfc15cc3615d0203347bd58cab819c36c48b0642a668f877","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-52a55290d17cd6c64c30055c","nextBlock":"b-2cbe2468e7a524119d2c5266","prerequisites":[{"id":"b-52a55290d17cd6c64c30055c","url":"#b-52a55290d17cd6c64c30055c","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-2cbe2468e7a524119d2c5266","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-2cbe2468e7a524119d2c5266","content":"先沿画面指出视觉特征进入哪里，再指出已有文字与下一词目标。连续说明展开 shifted target、EOS、PAD 和实际生成，LAB13 提供已保存的实验结果。理解本页可先把文字预测器作为输入输出模块；整数移位只演示目标构造，完整图像描述还需要图像条件与实际生成。","sha256":"aeaad29331bc2c94ac240ece646024336e7455de08d754b00ed438164663d9be","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ab410b4ca38fd3d426936f98","nextBlock":"b-14377e0b1fa48ceda8ad44c3","prerequisites":[{"id":"b-ab410b4ca38fd3d426936f98","url":"#b-ab410b4ca38fd3d426936f98","role":"section-title"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-14377e0b1fa48ceda8ad44c3","type":"figure","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-14377e0b1fa48ceda8ad44c3","content":"图内标签与数值：视觉编码器；图像 → 特征；两条合理描述；“两个人站在一起。”；“红衣人拿着篮球。”；文字预测器；允许看到的前文 + 视觉；红衣人 拿着；球；下一个目标；图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。\n\n图像先编码为视觉特征；视觉条件与已有文字一起进入文字预测器，前文从红衣人推进到红衣人拿着。\n\n- 视觉编码器\n- 图像 → 特征\n- 两条合理描述\n- “两个人站在一起。”\n- “红衣人拿着篮球。”\n- 文字预测器\n- 允许看到的前文 + 视觉\n- 红衣人 拿着\n- 球\n- 下一个目标\n- 图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。图像先编码为视觉特征；视觉条件与已有文字一起进入文字预测器，前文从红衣人推进到红衣人拿着。 [查看对应图解状态](/#/l3-r2-p51/1)[查看完整图解](/course/figures/l3-r2-p51-1.html)","sha256":"0e06fc8b3e1e98979520c451896ceecb0cb0e7c112adbed72ab8d8e12cd6a2e5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"图像先编码为视觉特征；视觉条件与已有文字一起进入文字预测器，前文从红衣人推进到红衣人拿着。 查看对应图解状态","labels":["视觉编码器","图像 → 特征","两条合理描述","“两个人站在一起。”","“红衣人拿着篮球。”","文字预测器","允许看到的前文 + 视觉","红衣人 拿着","球","下一个目标","图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。"],"description":"图像先编码为视觉特征；视觉条件与已有文字一起进入文字预测器，前文从红衣人推进到红衣人拿着。视觉编码器图像 → 特征两条合理描述“两个人站在一起。”“红衣人拿着篮球。”文字预测器允许看到的前文 + 视觉红衣人 拿着球下一个目标图文配对给出文字目标；页面展示训练接口与参考描述，没有展示模型生成结果。","evidenceIdentity":"图像先编码为视觉特征；视觉条件与已有文字一起进入文字预测器，前文从红衣人推进到红衣人拿着。 查看对应图解状态","images":[{"src":"/assets/display/512610a019c560ff0384e86a.webp","alt":""}]},"previousBlock":"b-2cbe2468e7a524119d2c5266","nextBlock":"b-9af35ea50d7361f697eb68e6","prerequisites":[{"id":"b-ab410b4ca38fd3d426936f98","url":"#b-ab410b4ca38fd3d426936f98","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9af35ea50d7361f697eb68e6","type":"h2","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-9af35ea50d7361f697eb68e6","content":"## 课堂讲述","sha256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-14377e0b1fa48ceda8ad44c3","nextBlock":"b-4d9e593b51c79d23f6b015f9","prerequisites":[{"id":"b-ab410b4ca38fd3d426936f98","url":"#b-ab410b4ca38fd3d426936f98","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-4d9e593b51c79d23f6b015f9","type":"p","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-4d9e593b51c79d23f6b015f9","content":"以图像描述为例，输出不再是固定类别，而是一段可变长度的文字。训练材料给出图像和描述，视觉特征与允许看到的前面文字一起，为后面的预测提供条件。\n注意，这里用的是图文配对监督，不是图像自己自动长出文字标签。不同描述可以关注不同内容：“两个人站在一起”概括人物和位置，“红衣人拿着球”指出动作主体。两句都能从当前画面得到支持。\n所以这个任务既要求视觉表示支持文字表达，也要求评价容纳多种合理描述。我们现在看完整的信息流和训练材料，不展开 RNN 或 Transformer 的内部机制。那部分留到第五讲，今天必须讲清的是：视觉信息从哪里进入，文字目标从哪里来，为什么流畅句子还要对得上画面。","sha256":"ad0b4788ea26b16e9d703df8dbbae2c66f4c88f9b18f35b738020ec6a87f043c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9af35ea50d7361f697eb68e6","nextBlock":"b-5b415a76297390ec24af06c3","prerequisites":[{"id":"b-9af35ea50d7361f697eb68e6","url":"#b-9af35ea50d7361f697eb68e6","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-5b415a76297390ec24af06c3","type":"h3","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-5b415a76297390ec24af06c3","content":"### 图像条件怎样进入一个可变长度答案","sha256":"2ab0f86797acfc0468ac83ce362620f38848977a36abd44ce780bd36873300f5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-conditioned-token","formulas":[],"previousBlock":"b-4d9e593b51c79d23f6b015f9","nextBlock":"b-d910beacbe22be8799463f90","prerequisites":[{"id":"b-9af35ea50d7361f697eb68e6","url":"#b-9af35ea50d7361f697eb68e6","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"},{"id":"b-d910beacbe22be8799463f90","url":"#b-d910beacbe22be8799463f90","relation":"same-reasoning"},{"id":"b-26e241fe0c2038f9fb90eb5e","url":"#b-26e241fe0c2038f9fb90eb5e","relation":"same-reasoning"},{"id":"b-2cc4348120b97b10fd16ffb8","url":"#b-2cc4348120b97b10fd16ffb8","relation":"same-reasoning"},{"id":"b-76cf5db89bafdbd550277f80","url":"#b-76cf5db89bafdbd550277f80","relation":"same-reasoning"}]},{"id":"b-d910beacbe22be8799463f90","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-d910beacbe22be8799463f90","content":"图像描述要建模的是 $p_\\theta(y_1,\\ldots,y_T\\mid I)$。链式法则将它分解为 $\\prod_t p_\\theta(y_t\\mid y_{<t},I)$：每一步都在图像和此前文字的条件下预测后续文字。这个分解是概率关系，不要求本讲先学完一种具体文字网络。","sha256":"73eb3160c6f026832b06ee63a280f8b54b7256670cc0346425cd8cb92efbd6e1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-conditioned-token","formulas":[{"latex":"p_\\theta(y_1,\\ldots,y_T\\mid I)","definitionContext":"图像描述要建模的是 $p_\\theta(y_1,\\ldots,y_T\\mid I)$。链式法则将它分解为 $\\prod_t p_\\theta(y_t\\mid y_{<t},I)$：每一步都在图像和此前文字的条件下预测后续文字。这个分解是概率关系，不要求本讲先学完一种具体文字网络。\n指着 P51 的图看三个位置：图像编码器把像素变成视觉特征；文字预测器同时接收视觉条件与已有文字；目标位置从“红衣人”后的“拿着”，推进到“红衣人拿着”后的“球”。如果删除视觉路径，模型仍可能补出语法合理的句子，却失去了判断画面到底是谁拿球的信息来源。\n训练图文配对提供目标句子，而不是图片自行生成唯一文字标签。一幅图可以被说成“两个人站在一起”，也可以被说成“红衣人拿着球”。前者省略衣色，后者聚焦动作主体，两句都可能真实。训练学习的是这种条件下的表达分布；教师还需要说明任务要求概括、细节描述还是完整枚举。\n推理时没有参考句子可偷看。模型从开始符号出发，输出一个词，再把这个词接到自己的上下文，直到结束符号或长度上限。完整的实验必须有实际图像、可学习参数、词目标、优化和这条自由生成过程。网页上推进的参考词是教学目标示意；实际生成与视觉对照放在 LAB13。"},{"latex":"\\prod_t p_\\theta(y_t\\mid y_{<t},I)","definitionContext":"图像描述要建模的是 $p_\\theta(y_1,\\ldots,y_T\\mid I)$。链式法则将它分解为 $\\prod_t p_\\theta(y_t\\mid y_{<t},I)$：每一步都在图像和此前文字的条件下预测后续文字。这个分解是概率关系，不要求本讲先学完一种具体文字网络。\n指着 P51 的图看三个位置：图像编码器把像素变成视觉特征；文字预测器同时接收视觉条件与已有文字；目标位置从“红衣人”后的“拿着”，推进到“红衣人拿着”后的“球”。如果删除视觉路径，模型仍可能补出语法合理的句子，却失去了判断画面到底是谁拿球的信息来源。\n训练图文配对提供目标句子，而不是图片自行生成唯一文字标签。一幅图可以被说成“两个人站在一起”，也可以被说成“红衣人拿着球”。前者省略衣色，后者聚焦动作主体，两句都可能真实。训练学习的是这种条件下的表达分布；教师还需要说明任务要求概括、细节描述还是完整枚举。\n推理时没有参考句子可偷看。模型从开始符号出发，输出一个词，再把这个词接到自己的上下文，直到结束符号或长度上限。完整的实验必须有实际图像、可学习参数、词目标、优化和这条自由生成过程。网页上推进的参考词是教学目标示意；实际生成与视觉对照放在 LAB13。"}],"previousBlock":"b-5b415a76297390ec24af06c3","nextBlock":"b-26e241fe0c2038f9fb90eb5e","prerequisites":[{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"},{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","relation":"same-reasoning"},{"id":"b-26e241fe0c2038f9fb90eb5e","url":"#b-26e241fe0c2038f9fb90eb5e","relation":"same-reasoning"},{"id":"b-2cc4348120b97b10fd16ffb8","url":"#b-2cc4348120b97b10fd16ffb8","relation":"same-reasoning"},{"id":"b-76cf5db89bafdbd550277f80","url":"#b-76cf5db89bafdbd550277f80","relation":"same-reasoning"}]},{"id":"b-26e241fe0c2038f9fb90eb5e","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-26e241fe0c2038f9fb90eb5e","content":"指着 P51 的图看三个位置：图像编码器把像素变成视觉特征；文字预测器同时接收视觉条件与已有文字；目标位置从“红衣人”后的“拿着”，推进到“红衣人拿着”后的“球”。如果删除视觉路径，模型仍可能补出语法合理的句子，却失去了判断画面到底是谁拿球的信息来源。","sha256":"963b8cb4593a865d67c3b515ce7a59eda4fde2d526bf1fab3eedc10e8f3bbe72","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-conditioned-token","formulas":[],"previousBlock":"b-d910beacbe22be8799463f90","nextBlock":"b-2cc4348120b97b10fd16ffb8","prerequisites":[{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"},{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","relation":"same-reasoning"},{"id":"b-d910beacbe22be8799463f90","url":"#b-d910beacbe22be8799463f90","relation":"same-reasoning"},{"id":"b-2cc4348120b97b10fd16ffb8","url":"#b-2cc4348120b97b10fd16ffb8","relation":"same-reasoning"},{"id":"b-76cf5db89bafdbd550277f80","url":"#b-76cf5db89bafdbd550277f80","relation":"same-reasoning"}]},{"id":"b-2cc4348120b97b10fd16ffb8","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-2cc4348120b97b10fd16ffb8","content":"训练图文配对提供目标句子，而不是图片自行生成唯一文字标签。一幅图可以被说成“两个人站在一起”，也可以被说成“红衣人拿着球”。前者省略衣色，后者聚焦动作主体，两句都可能真实。训练学习的是这种条件下的表达分布；教师还需要说明任务要求概括、细节描述还是完整枚举。","sha256":"38ac8dae0e03cd1374618f0f96fa050817f6b4ffb826dbe1ca1835086656d5be","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-conditioned-token","formulas":[],"previousBlock":"b-26e241fe0c2038f9fb90eb5e","nextBlock":"b-76cf5db89bafdbd550277f80","prerequisites":[{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"},{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","relation":"same-reasoning"},{"id":"b-d910beacbe22be8799463f90","url":"#b-d910beacbe22be8799463f90","relation":"same-reasoning"},{"id":"b-26e241fe0c2038f9fb90eb5e","url":"#b-26e241fe0c2038f9fb90eb5e","relation":"same-reasoning"},{"id":"b-76cf5db89bafdbd550277f80","url":"#b-76cf5db89bafdbd550277f80","relation":"same-reasoning"}]},{"id":"b-76cf5db89bafdbd550277f80","type":"p","scope":"caption-conditioned-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-76cf5db89bafdbd550277f80","content":"推理时没有参考句子可偷看。模型从开始符号出发，输出一个词，再把这个词接到自己的上下文，直到结束符号或长度上限。完整的实验必须有实际图像、可学习参数、词目标、优化和这条自由生成过程。网页上推进的参考词是教学目标示意；实际生成与视觉对照放在 LAB13。","sha256":"0c037e1d30329f16596e9cd66a94534e7997988977ea286a9927664b3809a313","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"图像条件怎样进入一个可变长度答案","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-conditioned-token","formulas":[],"previousBlock":"b-2cc4348120b97b10fd16ffb8","nextBlock":"b-6369f2a332a30216ca10e370","prerequisites":[{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"},{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","relation":"same-reasoning"},{"id":"b-d910beacbe22be8799463f90","url":"#b-d910beacbe22be8799463f90","relation":"same-reasoning"},{"id":"b-26e241fe0c2038f9fb90eb5e","url":"#b-26e241fe0c2038f9fb90eb5e","relation":"same-reasoning"},{"id":"b-2cc4348120b97b10fd16ffb8","url":"#b-2cc4348120b97b10fd16ffb8","relation":"same-reasoning"}]},{"id":"b-6369f2a332a30216ca10e370","type":"h4","scope":"caption-image-to-next-token","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-6369f2a332a30216ca10e370","content":"#### 图示条件与公式约定","sha256":"ef7f915c1ae63ed67bd23fecf050249273ba0efa8965d2c8d1c1c6b2d3178122","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-76cf5db89bafdbd550277f80","nextBlock":"b-878ff244e0d49b6a9e673c8f","prerequisites":[{"id":"b-5b415a76297390ec24af06c3","url":"#b-5b415a76297390ec24af06c3","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-878ff244e0d49b6a9e673c8f","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-878ff244e0d49b6a9e673c8f","content":"两条参考描述由课程作者写出。推进展示“红衣人”之后预测“拿着”，以及“红衣人拿着”之后预测“球”；这些是监督目标示例，并非实测解码结果。","sha256":"9f55c5cb4129e8e8b797be8286c4a119ea9d0bf7f9a8b67d2cf9b22501f57e96","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6369f2a332a30216ca10e370","nextBlock":"b-59a13d1d954534a54208c673","prerequisites":[{"id":"b-6369f2a332a30216ca10e370","url":"#b-6369f2a332a30216ca10e370","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-59a13d1d954534a54208c673","type":"h4","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-59a13d1d954534a54208c673","content":"#### 这条线索的展开与后续阅读","sha256":"48b8229f7559b36ed4d5628416e3991e5c73293fbce43374db67386c8736f3bc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-878ff244e0d49b6a9e673c8f","nextBlock":"b-b8f7ec9042ba6e44ae10e77e","prerequisites":[{"id":"b-6369f2a332a30216ca10e370","url":"#b-6369f2a332a30216ca10e370","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-b8f7ec9042ba6e44ae10e77e","type":"ul","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-b8f7ec9042ba6e44ae10e77e","content":"- [架构、目标与解码必须分开｜一起算一个完整例子](/supplement.html?return=l3-r2-p51/0#/l3-architecture-objective-decoding-example/0) 第五讲预备阅读\n- [架构、目标与解码必须分开｜怎样检查这项解释](/supplement.html?return=l3-r2-p51/0#/l3-architecture-objective-decoding-evidence/0) 第七讲相关预备阅读\n- [让图像成为下一词预测的条件](/supplement.html?return=l3-r2-p51/0#/l3-captioning/0) 本页说明\n- [描述任务把视觉接到可变长度输出｜问题从哪里出现](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-problem/0) 本页说明\n- [描述任务把视觉接到可变长度输出｜核心改动怎样工作](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-mechanism/0) 本页说明\n- [描述任务把视觉接到可变长度输出｜一起算一个完整例子](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-example/0) 本页说明\n- [描述任务把视觉接到可变长度输出｜什么结论还不能推出](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-boundary/0) 本页说明\n- [训练前缀与生成前缀不同｜问题从哪里出现](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-problem/0) 第五讲预备阅读\n- [训练前缀与生成前缀不同｜核心改动怎样工作](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-mechanism/0) 第五讲预备阅读\n- [训练前缀与生成前缀不同｜一起算一个完整例子](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-example/0) 第七讲相关预备阅读\n- [训练前缀与生成前缀不同｜怎样检查这项解释](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-evidence/0) 第七讲相关预备阅读\n- [训练前缀与生成前缀不同｜什么结论还不能推出](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-boundary/0) 第七讲相关预备阅读","sha256":"d6b83489df8601fc692dca4e701aa80750a0218408fbf8ac355ca7a8adee4249","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-59a13d1d954534a54208c673","nextBlock":"b-cb4c7be5b7131744ae04a8e5","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-cb4c7be5b7131744ae04a8e5","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-cb4c7be5b7131744ae04a8e5","content":"- [架构、目标与解码必须分开｜一起算一个完整例子](/supplement.html?return=l3-r2-p51/0#/l3-architecture-objective-decoding-example/0) 第五讲预备阅读","sha256":"06390678b1b9671244bcf735621682f5edfd65ca00ca25fa2011bf82d5b235f4","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b8f7ec9042ba6e44ae10e77e","nextBlock":"b-b7df37fc48c089cfe2569a8f","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-b7df37fc48c089cfe2569a8f","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-b7df37fc48c089cfe2569a8f","content":"- [架构、目标与解码必须分开｜怎样检查这项解释](/supplement.html?return=l3-r2-p51/0#/l3-architecture-objective-decoding-evidence/0) 第七讲相关预备阅读","sha256":"994b2a08a8cadd1cc3df33daa7df50aba5d07c339e218a42d7cce67b43964fe6","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-cb4c7be5b7131744ae04a8e5","nextBlock":"b-871676744729abf7dec49f40","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-871676744729abf7dec49f40","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-871676744729abf7dec49f40","content":"- [让图像成为下一词预测的条件](/supplement.html?return=l3-r2-p51/0#/l3-captioning/0) 本页说明","sha256":"2f42fe05aebc54159c3c3af4649940c3c73dfaaf6d4bf90ac7168ec5a626ca08","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b7df37fc48c089cfe2569a8f","nextBlock":"b-fc16bf4219289e5738993935","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-fc16bf4219289e5738993935","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-fc16bf4219289e5738993935","content":"- [描述任务把视觉接到可变长度输出｜问题从哪里出现](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-problem/0) 本页说明","sha256":"30c75f370a12cfa3e7ddb387dab529922355c84b1acd04f3cc00407380f879e1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-871676744729abf7dec49f40","nextBlock":"b-6fef75ac33b658bc054dc759","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-6fef75ac33b658bc054dc759","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-6fef75ac33b658bc054dc759","content":"- [描述任务把视觉接到可变长度输出｜核心改动怎样工作](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-mechanism/0) 本页说明","sha256":"16bc21ee8d6fe340ac60513d6125f0f5190a6d07566276dfa218beedddc0d9b3","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-fc16bf4219289e5738993935","nextBlock":"b-2cd73f242adcad72ad553ab2","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-2cd73f242adcad72ad553ab2","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-2cd73f242adcad72ad553ab2","content":"- [描述任务把视觉接到可变长度输出｜一起算一个完整例子](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-example/0) 本页说明","sha256":"adca7a5d4689903412db69da89606445c541264728d3c505a705f275d08fd7d7","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6fef75ac33b658bc054dc759","nextBlock":"b-f44ed46f57462b6fb5289094","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-f44ed46f57462b6fb5289094","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-f44ed46f57462b6fb5289094","content":"- [描述任务把视觉接到可变长度输出｜什么结论还不能推出](/supplement.html?return=l3-r2-p51/0#/l3-caption-history-boundary/0) 本页说明","sha256":"c48baab888f8b3eec0764bc35694f23349c16c01f8c12ab5fc3efe8a94c69740","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2cd73f242adcad72ad553ab2","nextBlock":"b-a5aba144db2c50186c498741","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-a5aba144db2c50186c498741","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-a5aba144db2c50186c498741","content":"- [训练前缀与生成前缀不同｜问题从哪里出现](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-problem/0) 第五讲预备阅读","sha256":"89a322d64437b3595d77045707a818d41f648d725ac24e3f284a6d8f51707cc3","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f44ed46f57462b6fb5289094","nextBlock":"b-21714ca5e3e3202ccab6c32a","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-21714ca5e3e3202ccab6c32a","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-21714ca5e3e3202ccab6c32a","content":"- [训练前缀与生成前缀不同｜核心改动怎样工作](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-mechanism/0) 第五讲预备阅读","sha256":"2c172d358ceceb1a46e47a24cc98a2615f53e2848b17d6551df0e471d9a3ac52","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a5aba144db2c50186c498741","nextBlock":"b-8e3f623d987cbe3593ecea83","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-8e3f623d987cbe3593ecea83","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-8e3f623d987cbe3593ecea83","content":"- [训练前缀与生成前缀不同｜一起算一个完整例子](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-example/0) 第七讲相关预备阅读","sha256":"8fab6c6d454d29102d7d5a31edd6b6b7cf7d07a3daef7ad1dd882032726099c9","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-21714ca5e3e3202ccab6c32a","nextBlock":"b-3d96dbf8c82c917088121158","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-3d96dbf8c82c917088121158","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-3d96dbf8c82c917088121158","content":"- [训练前缀与生成前缀不同｜怎样检查这项解释](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-evidence/0) 第七讲相关预备阅读","sha256":"31dd0c4957c4b012df377041af0122fadd7ef43ecc33fb3ae9cf182150a4d60e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8e3f623d987cbe3593ecea83","nextBlock":"b-0244a6799a6e12d1411f8175","prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-0244a6799a6e12d1411f8175","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-0244a6799a6e12d1411f8175","content":"- [训练前缀与生成前缀不同｜什么结论还不能推出](/supplement.html?return=l3-r2-p51/0#/l3-caption-training-boundary/0) 第七讲相关预备阅读","sha256":"ca368380e67807da96d28a3d42bac8a7130788bfad138122839a7f170a2e9e72","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-image-to-next-token","title":"沿视觉条件找到下一词目标","number":1},"sectionTitle":"沿视觉条件找到下一词目标","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3d96dbf8c82c917088121158","nextBlock":null,"prerequisites":[{"id":"b-59a13d1d954534a54208c673","url":"#b-59a13d1d954534a54208c673","role":"section-title"},{"id":"b-2cbe2468e7a524119d2c5266","url":"#b-2cbe2468e7a524119d2c5266","role":"problem-context"}],"relatedBlocks":[{"id":"b-14377e0b1fa48ceda8ad44c3","url":"#b-14377e0b1fa48ceda8ad44c3","relation":"figure-and-explanation"}]},{"id":"b-d3ade6c5e1c5567a00bc21f6","type":"h2","scope":"caption-targets-and-stopping","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-d3ade6c5e1c5567a00bc21f6","content":"## 对齐前缀、目标与有效损失位置","sha256":"fc9319b0241f98487906696326bd4f179f5934778dc2e4431c672c006a64ae7d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"对齐前缀、目标与有效损失位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-288866af8ebe58e22e4d516a","prerequisites":[{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-288866af8ebe58e22e4d516a","type":"p","scope":"caption-targets-and-stopping","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-288866af8ebe58e22e4d516a","content":"图像与已有文字提供条件；下一步要明确每个位置到底预测什么。","sha256":"e7f208e0502d7d212cffc3b3b3b852d67f8f423ae0e2ada28e23252b28e681c9","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"对齐前缀、目标与有效损失位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d3ade6c5e1c5567a00bc21f6","nextBlock":"b-c586b2c85bf8e363de087cee","prerequisites":[{"id":"b-d3ade6c5e1c5567a00bc21f6","url":"#b-d3ade6c5e1c5567a00bc21f6","role":"section-title"}],"relatedBlocks":[]},{"id":"b-c586b2c85bf8e363de087cee","type":"h3","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-c586b2c85bf8e363de087cee","content":"### 训练前缀、EOS 与 PAD：损失究竟算在哪些位置","sha256":"8c2505555011d4c3abd71b059fb64a24aae2fe9370ea4bc01db532cb1c16fba9","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-shift-and-stop","formulas":[],"previousBlock":"b-288866af8ebe58e22e4d516a","nextBlock":"b-e462382727de092875ec35b4","prerequisites":[{"id":"b-d3ade6c5e1c5567a00bc21f6","url":"#b-d3ade6c5e1c5567a00bc21f6","role":"section-title"},{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[{"id":"b-e462382727de092875ec35b4","url":"#b-e462382727de092875ec35b4","relation":"same-reasoning"},{"id":"b-a8af80f9dfed62aabe85f456","url":"#b-a8af80f9dfed62aabe85f456","relation":"same-reasoning"},{"id":"b-f018399dadfb59081776c116","url":"#b-f018399dadfb59081776c116","relation":"same-reasoning"},{"id":"b-977d04e10b68031b8aa7a8cc","url":"#b-977d04e10b68031b8aa7a8cc","relation":"same-reasoning"},{"id":"b-adba7b27272722ca00ece129","url":"#b-adba7b27272722ca00ece129","relation":"same-reasoning"}]},{"id":"b-e462382727de092875ec35b4","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-e462382727de092875ec35b4","content":"假设一条参考句为 `a red square`，词序列写成 `[BOS,a,red,square,EOS]`。训练输入去掉最后一项，目标去掉第一项。两者一一对齐：","sha256":"dd0cbe5a9f571aff92f5177d64ed0ecef2a16c1713f740eea8cb5312f7a9f650","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-shift-and-stop","formulas":[],"previousBlock":"b-c586b2c85bf8e363de087cee","nextBlock":"b-a8af80f9dfed62aabe85f456","prerequisites":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","role":"section-title"},{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","relation":"same-reasoning"},{"id":"b-a8af80f9dfed62aabe85f456","url":"#b-a8af80f9dfed62aabe85f456","relation":"same-reasoning"},{"id":"b-f018399dadfb59081776c116","url":"#b-f018399dadfb59081776c116","relation":"same-reasoning"},{"id":"b-977d04e10b68031b8aa7a8cc","url":"#b-977d04e10b68031b8aa7a8cc","relation":"same-reasoning"},{"id":"b-adba7b27272722ca00ece129","url":"#b-adba7b27272722ca00ece129","relation":"same-reasoning"}]},{"id":"b-a8af80f9dfed62aabe85f456","type":"table","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-a8af80f9dfed62aabe85f456","content":"| 已允许看到的文字 | 当前目标 |\n| --- | --- |\n| BOS | a |\n| BOS a | red |\n| BOS a red | square |\n| BOS a red square | EOS |","sha256":"7cf82523725f8dc1dbfc45a269f6e423480bce0858c9bf86dcb2bc5f68cfb9ab","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-shift-and-stop","formulas":[],"table":{"headers":["已允许看到的文字","当前目标"],"rows":[["已允许看到的文字","当前目标"],["BOS","a"],["BOS a","red"],["BOS a red","square"],["BOS a red square","EOS"]]},"previousBlock":"b-e462382727de092875ec35b4","nextBlock":"b-f018399dadfb59081776c116","prerequisites":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","role":"section-title"},{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","relation":"same-reasoning"},{"id":"b-e462382727de092875ec35b4","url":"#b-e462382727de092875ec35b4","relation":"same-reasoning"},{"id":"b-f018399dadfb59081776c116","url":"#b-f018399dadfb59081776c116","relation":"same-reasoning"},{"id":"b-977d04e10b68031b8aa7a8cc","url":"#b-977d04e10b68031b8aa7a8cc","relation":"same-reasoning"},{"id":"b-adba7b27272722ca00ece129","url":"#b-adba7b27272722ca00ece129","relation":"same-reasoning"}]},{"id":"b-f018399dadfb59081776c116","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-f018399dadfb59081776c116","content":"如果批中还有更长句子，就给短句末尾补 PAD，使张量长度一致。PAD 只是存储约定，不是要生成的文字；EOS 却是有效的停止目标。掩掉 PAD 时把 EOS 一起掩掉，会抹去何时结束的监督。","sha256":"43952b0c901e562451395edc1a8bdeef87074554a5ad6cc25045b1d31c451f16","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-shift-and-stop","formulas":[],"previousBlock":"b-a8af80f9dfed62aabe85f456","nextBlock":"b-977d04e10b68031b8aa7a8cc","prerequisites":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","role":"section-title"},{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","relation":"same-reasoning"},{"id":"b-e462382727de092875ec35b4","url":"#b-e462382727de092875ec35b4","relation":"same-reasoning"},{"id":"b-a8af80f9dfed62aabe85f456","url":"#b-a8af80f9dfed62aabe85f456","relation":"same-reasoning"},{"id":"b-977d04e10b68031b8aa7a8cc","url":"#b-977d04e10b68031b8aa7a8cc","relation":"same-reasoning"},{"id":"b-adba7b27272722ca00ece129","url":"#b-adba7b27272722ca00ece129","relation":"same-reasoning"}]},{"id":"b-977d04e10b68031b8aa7a8cc","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-977d04e10b68031b8aa7a8cc","content":"词损失可写为 $L=-\\sum_{i,t}m_{it}\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})/\\sum_{i,t}m_{it}$，其中 $m_{it}=1$ 表示目标不是 PAD。分母按有效词计数，不按补齐后的格数计数。LAB13 的 `prefix=Y[:,:-1]`、`targets=Y[:,1:]` 和 `ignore_index=PAD` 对应这三个选择。","sha256":"abec1e77f4ee30f312e664a3c45a315e4bfe246e526b1dd7bb578311d2ada770","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-shift-and-stop","formulas":[{"latex":"L=-\\sum_{i,t}m_{it}\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})/\\sum_{i,t}m_{it}","definitionContext":"假设一条参考句为 `a red square`，词序列写成 `[BOS,a,red,square,EOS]`。训练输入去掉最后一项，目标去掉第一项。两者一一对齐：\n如果批中还有更长句子，就给短句末尾补 PAD，使张量长度一致。PAD 只是存储约定，不是要生成的文字；EOS 却是有效的停止目标。掩掉 PAD 时把 EOS 一起掩掉，会抹去何时结束的监督。\n词损失可写为 $L=-\\sum_{i,t}m_{it}\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})/\\sum_{i,t}m_{it}$，其中 $m_{it}=1$ 表示目标不是 PAD。分母按有效词计数，不按补齐后的格数计数。LAB13 的 `prefix=Y[:,:-1]`、`targets=Y[:,1:]` 和 `ignore_index=PAD` 对应这三个选择。\n训练时喂真实前缀叫 teacher forcing。推理时前缀来自模型自己：若先误说 blue，后续就必须在 blue 的条件下继续，不能偷偷换回 red。因此保留集的逐词交叉熵与自由生成的整句正确率要分别读；两者评价的输入条件不同。"},{"latex":"m_{it}=1","definitionContext":"假设一条参考句为 `a red square`，词序列写成 `[BOS,a,red,square,EOS]`。训练输入去掉最后一项，目标去掉第一项。两者一一对齐：\n如果批中还有更长句子，就给短句末尾补 PAD，使张量长度一致。PAD 只是存储约定，不是要生成的文字；EOS 却是有效的停止目标。掩掉 PAD 时把 EOS 一起掩掉，会抹去何时结束的监督。\n词损失可写为 $L=-\\sum_{i,t}m_{it}\\log p_\\theta(y_{it}\\mid I_i,y_{i,<t})/\\sum_{i,t}m_{it}$，其中 $m_{it}=1$ 表示目标不是 PAD。分母按有效词计数，不按补齐后的格数计数。LAB13 的 `prefix=Y[:,:-1]`、`targets=Y[:,1:]` 和 `ignore_index=PAD` 对应这三个选择。\n训练时喂真实前缀叫 teacher forcing。推理时前缀来自模型自己：若先误说 blue，后续就必须在 blue 的条件下继续，不能偷偷换回 red。因此保留集的逐词交叉熵与自由生成的整句正确率要分别读；两者评价的输入条件不同。"}],"previousBlock":"b-f018399dadfb59081776c116","nextBlock":"b-adba7b27272722ca00ece129","prerequisites":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","role":"section-title"},{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","relation":"same-reasoning"},{"id":"b-e462382727de092875ec35b4","url":"#b-e462382727de092875ec35b4","relation":"same-reasoning"},{"id":"b-a8af80f9dfed62aabe85f456","url":"#b-a8af80f9dfed62aabe85f456","relation":"same-reasoning"},{"id":"b-f018399dadfb59081776c116","url":"#b-f018399dadfb59081776c116","relation":"same-reasoning"},{"id":"b-adba7b27272722ca00ece129","url":"#b-adba7b27272722ca00ece129","relation":"same-reasoning"}]},{"id":"b-adba7b27272722ca00ece129","type":"p","scope":"caption-shift-and-stop","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-adba7b27272722ca00ece129","content":"训练时喂真实前缀叫 teacher forcing。推理时前缀来自模型自己：若先误说 blue，后续就必须在 blue 的条件下继续，不能偷偷换回 red。因此保留集的逐词交叉熵与自由生成的整句正确率要分别读；两者评价的输入条件不同。","sha256":"37cf03728db1dab4711e27ff47809b2be238f681a3a44e01bcb88c5744536edc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-targets-and-stopping","title":"对齐前缀、目标与有效损失位置","number":2},"sectionTitle":"训练前缀、EOS 与 PAD：损失究竟算在哪些位置","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-shift-and-stop","formulas":[],"previousBlock":"b-977d04e10b68031b8aa7a8cc","nextBlock":null,"prerequisites":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","role":"section-title"},{"id":"b-288866af8ebe58e22e4d516a","url":"#b-288866af8ebe58e22e4d516a","role":"problem-context"}],"relatedBlocks":[{"id":"b-c586b2c85bf8e363de087cee","url":"#b-c586b2c85bf8e363de087cee","relation":"same-reasoning"},{"id":"b-e462382727de092875ec35b4","url":"#b-e462382727de092875ec35b4","relation":"same-reasoning"},{"id":"b-a8af80f9dfed62aabe85f456","url":"#b-a8af80f9dfed62aabe85f456","relation":"same-reasoning"},{"id":"b-f018399dadfb59081776c116","url":"#b-f018399dadfb59081776c116","relation":"same-reasoning"},{"id":"b-977d04e10b68031b8aa7a8cc","url":"#b-977d04e10b68031b8aa7a8cc","relation":"same-reasoning"}]},{"id":"b-3275b7767665ca864c0be4bc","type":"h2","scope":"caption-lab13-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-3275b7767665ca864c0be4bc","content":"## 先读固定前缀下的视觉对照结果","sha256":"fb3e0a73bdc974d08c0f803a7950c614108d9ad4756f49423fec0d17a65909ba","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"先读固定前缀下的视觉对照结果","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-ac3bb16f7db137b3d2d65f15","prerequisites":[{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ac3bb16f7db137b3d2d65f15","type":"p","scope":"caption-lab13-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-ac3bb16f7db137b3d2d65f15","content":"词级损失描述训练；自由生成和视觉干预才能检查当前任务中的视觉作用。","sha256":"58a36406b8aa4fa2e3699cf54bcd6ec4b2ad1a9042b38e9133e9856accd0c224","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"先读固定前缀下的视觉对照结果","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3275b7767665ca864c0be4bc","nextBlock":"b-2d17b377c21a89d013046fac","prerequisites":[{"id":"b-3275b7767665ca864c0be4bc","url":"#b-3275b7767665ca864c0be4bc","role":"section-title"}],"relatedBlocks":[]},{"id":"b-2d17b377c21a89d013046fac","type":"h3","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-2d17b377c21a89d013046fac","content":"### 已保存的条件生成实验：视觉输入确实改变了句子","sha256":"8254cd6b23c829245d1f704a82870a58d61049cf00ad79ec44e348371e74dbf3","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[],"previousBlock":"b-ac3bb16f7db137b3d2d65f15","nextBlock":"b-42556ad4acb0bbe952b01cab","prerequisites":[{"id":"b-3275b7767665ca864c0be4bc","url":"#b-3275b7767665ca864c0be4bc","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-42556ad4acb0bbe952b01cab","url":"#b-42556ad4acb0bbe952b01cab","relation":"same-reasoning"},{"id":"b-1b5258cda22c24c30e82c5ea","url":"#b-1b5258cda22c24c30e82c5ea","relation":"same-reasoning"},{"id":"b-84aad620e02cc718409a4995","url":"#b-84aad620e02cc718409a4995","relation":"same-reasoning"},{"id":"b-ff901967d6958ea8eb362fd6","url":"#b-ff901967d6958ea8eb362fd6","relation":"same-reasoning"},{"id":"b-65dab675304107e641dde759","url":"#b-65dab675304107e641dde759","relation":"same-reasoning"},{"id":"b-15c07ba6586c133142f20a91","url":"#b-15c07ba6586c133142f20a91","relation":"same-reasoning"}]},{"id":"b-42556ad4acb0bbe952b01cab","type":"p","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-42556ad4acb0bbe952b01cab","content":"LAB13 当前保存的是 2026-09-22 本地 CPU 基准，当前整理没有重新训练。输入为真实渲染的 $32\\times32$ RGB 图像；训练 720 张，保留 180 张。颜色、形状、位置共有 18 种组合，两侧都包含这些组合，但保留图由独立随机渲染产生、像素哈希不重复。这检验新渲染泛化，不是未见组合泛化。","sha256":"c1e2bdbd6a16066dce6e56451311b7266e3a63bc810800f91e36da038f829364","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[{"latex":"32\\times32","definitionContext":"LAB13 当前保存的是 2026-09-22 本地 CPU 基准，当前整理没有重新训练。输入为真实渲染的 $32\\times32$ RGB 图像；训练 720 张，保留 180 张。颜色、形状、位置共有 18 种组合，两侧都包含这些组合，但保留图由独立随机渲染产生、像素哈希不重复。这检验新渲染泛化，不是未见组合泛化。\nCNN 输出 48 维视觉向量；文字单位有 24 维嵌入；实验采用一个小型 GRU 文字模块。今天把它作为已明确输入输出的实现，内部循环状态留第五讲。视觉向量确实参与每一步预测，并用于初始化文字模块；真实颜色或位置标签只用于构造监督，不作为推理输入。\n固定前缀 `BOS a`，红方块左侧图生成 `a red square on left`，蓝圆右侧图生成 `a blue circle on right`。这里正面成立的判断是：这个有限任务中，训练后的输出使用了正确视觉条件。两种搜索同分，不支持 beam 比 greedy 更好；零图像是分布外条件，错配对照则进一步隔离了图像对应关系。自然图像有多种合理描述，不能把这里唯一规范句子的精确匹配直接移作通用 caption 评价。\n![固定前缀下的真实生成](/assets/display/9ebe76d49a449b9940b6470f.webp)\n图片与数字均来自已保存基准，不是把数据生成器的目标文字重新印成“模型回答”。"}],"previousBlock":"b-2d17b377c21a89d013046fac","nextBlock":"b-1b5258cda22c24c30e82c5ea","prerequisites":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","relation":"same-reasoning"},{"id":"b-1b5258cda22c24c30e82c5ea","url":"#b-1b5258cda22c24c30e82c5ea","relation":"same-reasoning"},{"id":"b-84aad620e02cc718409a4995","url":"#b-84aad620e02cc718409a4995","relation":"same-reasoning"},{"id":"b-ff901967d6958ea8eb362fd6","url":"#b-ff901967d6958ea8eb362fd6","relation":"same-reasoning"},{"id":"b-65dab675304107e641dde759","url":"#b-65dab675304107e641dde759","relation":"same-reasoning"},{"id":"b-15c07ba6586c133142f20a91","url":"#b-15c07ba6586c133142f20a91","relation":"same-reasoning"}]},{"id":"b-1b5258cda22c24c30e82c5ea","type":"p","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-1b5258cda22c24c30e82c5ea","content":"CNN 输出 48 维视觉向量；文字单位有 24 维嵌入；实验采用一个小型 GRU 文字模块。今天把它作为已明确输入输出的实现，内部循环状态留第五讲。视觉向量确实参与每一步预测，并用于初始化文字模块；真实颜色或位置标签只用于构造监督，不作为推理输入。","sha256":"d6e2fdeea4c473a4ad3b0b4a0a761eeafdcdf897215012754c0cfc5db9ebb042","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[],"previousBlock":"b-42556ad4acb0bbe952b01cab","nextBlock":"b-84aad620e02cc718409a4995","prerequisites":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","relation":"same-reasoning"},{"id":"b-42556ad4acb0bbe952b01cab","url":"#b-42556ad4acb0bbe952b01cab","relation":"same-reasoning"},{"id":"b-84aad620e02cc718409a4995","url":"#b-84aad620e02cc718409a4995","relation":"same-reasoning"},{"id":"b-ff901967d6958ea8eb362fd6","url":"#b-ff901967d6958ea8eb362fd6","relation":"same-reasoning"},{"id":"b-65dab675304107e641dde759","url":"#b-65dab675304107e641dde759","relation":"same-reasoning"},{"id":"b-15c07ba6586c133142f20a91","url":"#b-15c07ba6586c133142f20a91","relation":"same-reasoning"}]},{"id":"b-84aad620e02cc718409a4995","type":"table","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-84aad620e02cc718409a4995","content":"| 同一组 180 张保留图 | 整句精确匹配 |\n| --- | --- |\n| 正确图像，greedy 生成 | 100% |\n| 正确图像，beam 宽度 3 | 100% |\n| 同一模型，像素置零 | 5.56% |\n| 同一模型，图像循环错配 | 0% |","sha256":"6198a4971317db245f284fd359e91844808af55181f829edcbe77f7009c63809","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[],"table":{"headers":["同一组 180 张保留图","整句精确匹配"],"rows":[["同一组 180 张保留图","整句精确匹配"],["正确图像，greedy 生成","100%"],["正确图像，beam 宽度 3","100%"],["同一模型，像素置零","5.56%"],["同一模型，图像循环错配","0%"]]},"previousBlock":"b-1b5258cda22c24c30e82c5ea","nextBlock":"b-ff901967d6958ea8eb362fd6","prerequisites":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","relation":"same-reasoning"},{"id":"b-42556ad4acb0bbe952b01cab","url":"#b-42556ad4acb0bbe952b01cab","relation":"same-reasoning"},{"id":"b-1b5258cda22c24c30e82c5ea","url":"#b-1b5258cda22c24c30e82c5ea","relation":"same-reasoning"},{"id":"b-ff901967d6958ea8eb362fd6","url":"#b-ff901967d6958ea8eb362fd6","relation":"same-reasoning"},{"id":"b-65dab675304107e641dde759","url":"#b-65dab675304107e641dde759","relation":"same-reasoning"},{"id":"b-15c07ba6586c133142f20a91","url":"#b-15c07ba6586c133142f20a91","relation":"same-reasoning"}]},{"id":"b-ff901967d6958ea8eb362fd6","type":"p","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-ff901967d6958ea8eb362fd6","content":"固定前缀 `BOS a`，红方块左侧图生成 `a red square on left`，蓝圆右侧图生成 `a blue circle on right`。这里正面成立的判断是：这个有限任务中，训练后的输出使用了正确视觉条件。两种搜索同分，不支持 beam 比 greedy 更好；零图像是分布外条件，错配对照则进一步隔离了图像对应关系。自然图像有多种合理描述，不能把这里唯一规范句子的精确匹配直接移作通用 caption 评价。","sha256":"1f615531076a140376b37f33392e5543bedc80c8ddb1b9b6483dd6d7e4d5f06b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[],"previousBlock":"b-84aad620e02cc718409a4995","nextBlock":"b-65dab675304107e641dde759","prerequisites":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","relation":"same-reasoning"},{"id":"b-42556ad4acb0bbe952b01cab","url":"#b-42556ad4acb0bbe952b01cab","relation":"same-reasoning"},{"id":"b-1b5258cda22c24c30e82c5ea","url":"#b-1b5258cda22c24c30e82c5ea","relation":"same-reasoning"},{"id":"b-84aad620e02cc718409a4995","url":"#b-84aad620e02cc718409a4995","relation":"same-reasoning"},{"id":"b-65dab675304107e641dde759","url":"#b-65dab675304107e641dde759","relation":"same-reasoning"},{"id":"b-15c07ba6586c133142f20a91","url":"#b-15c07ba6586c133142f20a91","relation":"same-reasoning"}]},{"id":"b-65dab675304107e641dde759","type":"p","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-65dab675304107e641dde759","content":"![固定前缀下的真实生成](/assets/display/9ebe76d49a449b9940b6470f.webp)","sha256":"f6aedf109a28f615cb59b69833b6929e30ff48023b1eabc37990766d7cfd0565","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[],"previousBlock":"b-ff901967d6958ea8eb362fd6","nextBlock":"b-15c07ba6586c133142f20a91","prerequisites":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","relation":"same-reasoning"},{"id":"b-42556ad4acb0bbe952b01cab","url":"#b-42556ad4acb0bbe952b01cab","relation":"same-reasoning"},{"id":"b-1b5258cda22c24c30e82c5ea","url":"#b-1b5258cda22c24c30e82c5ea","relation":"same-reasoning"},{"id":"b-84aad620e02cc718409a4995","url":"#b-84aad620e02cc718409a4995","relation":"same-reasoning"},{"id":"b-ff901967d6958ea8eb362fd6","url":"#b-ff901967d6958ea8eb362fd6","relation":"same-reasoning"},{"id":"b-15c07ba6586c133142f20a91","url":"#b-15c07ba6586c133142f20a91","relation":"same-reasoning"}]},{"id":"b-15c07ba6586c133142f20a91","type":"p","scope":"caption-visual-controls","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-15c07ba6586c133142f20a91","content":"图片与数字均来自已保存基准，不是把数据生成器的目标文字重新印成“模型回答”。","sha256":"38fc170e9722124fda4bcec90baa5dd1ee7e03550b153f899912a90c1fbb4ce0","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"caption-visual-conditioning","title":"图像怎样约束一句新生成的描述？"},"page":{"id":"caption-lab13-visual-controls","title":"先读固定前缀下的视觉对照结果","number":3},"sectionTitle":"已保存的条件生成实验：视觉输入确实改变了句子","currentQuestion":"从训练目标到自由生成，怎样检查模型实际使用了图像？","sourceSlides":["l3-r2-p51"],"knowledgeBlock":"caption-visual-controls","formulas":[],"previousBlock":"b-65dab675304107e641dde759","nextBlock":null,"prerequisites":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","role":"section-title"},{"id":"b-ac3bb16f7db137b3d2d65f15","url":"#b-ac3bb16f7db137b3d2d65f15","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d17b377c21a89d013046fac","url":"#b-2d17b377c21a89d013046fac","relation":"same-reasoning"},{"id":"b-42556ad4acb0bbe952b01cab","url":"#b-42556ad4acb0bbe952b01cab","relation":"same-reasoning"},{"id":"b-1b5258cda22c24c30e82c5ea","url":"#b-1b5258cda22c24c30e82c5ea","relation":"same-reasoning"},{"id":"b-84aad620e02cc718409a4995","url":"#b-84aad620e02cc718409a4995","relation":"same-reasoning"},{"id":"b-ff901967d6958ea8eb362fd6","url":"#b-ff901967d6958ea8eb362fd6","relation":"same-reasoning"},{"id":"b-65dab675304107e641dde759","url":"#b-65dab675304107e641dde759","relation":"same-reasoning"}]},{"id":"b-39315a7e537d87cbbe0dfbcc","type":"h2","scope":"caption-search-probability-extension","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-39315a7e537d87cbbe0dfbcc","content":"## 选读：比较局部选择与完整路径概率","sha256":"d85ecc998b83d49ca5e340a8a1bc282697016fd0f5fccf3e6f80311abb670e76","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-93a7174a2a98e127794dcf5d","prerequisites":[{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-93a7174a2a98e127794dcf5d","type":"p","scope":"caption-search-probability-extension","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-93a7174a2a98e127794dcf5d","content":"Caption 已经使用逐词生成；这段保留为第四讲生成与第七讲搜索的预备阅读。","sha256":"2e178b7581e2f95ec9e21289f8163e4363c4b209d9cad76b457162d4732d077c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-39315a7e537d87cbbe0dfbcc","nextBlock":"b-318eba2140cce74164a6d0b1","prerequisites":[{"id":"b-39315a7e537d87cbbe0dfbcc","url":"#b-39315a7e537d87cbbe0dfbcc","role":"section-title"}],"relatedBlocks":[]},{"id":"b-318eba2140cce74164a6d0b1","type":"h3","scope":"caption-decoding-later","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-318eba2140cce74164a6d0b1","content":"### 选读：局部最高概率为何不保证整句最高概率","sha256":"44317b8cd70503c8912fc895871b14162b2f4220d9fe401770b3bd9f948c602d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：局部最高概率为何不保证整句最高概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":"caption-decoding-later","formulas":[],"previousBlock":"b-93a7174a2a98e127794dcf5d","nextBlock":"b-83a4df7935e400f89d3aa427","prerequisites":[{"id":"b-39315a7e537d87cbbe0dfbcc","url":"#b-39315a7e537d87cbbe0dfbcc","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[{"id":"b-83a4df7935e400f89d3aa427","url":"#b-83a4df7935e400f89d3aa427","relation":"same-reasoning"},{"id":"b-b7dda3daa921ecbce135ae19","url":"#b-b7dda3daa921ecbce135ae19","relation":"same-reasoning"},{"id":"b-fca31d6d6facb0c582ccf434","url":"#b-fca31d6d6facb0c582ccf434","relation":"same-reasoning"},{"id":"b-ef4ba79b1b52a3072a01a9fc","url":"#b-ef4ba79b1b52a3072a01a9fc","relation":"same-reasoning"}]},{"id":"b-83a4df7935e400f89d3aa427","type":"p","scope":"caption-decoding-later","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-83a4df7935e400f89d3aa427","content":"这段作为第四讲生成和第七讲搜索的预备阅读；第三讲只需知道推理会接续自己的输出。令首词 A/B 概率为 0.6/0.4；A 后 X/Y 为 0.51/0.49；B 后 X/Y 为 0.9/0.1；下一步都以概率 1 输出 EOS。每层出边和为 1。","sha256":"c69ff8d382b78b38f116e433914d074c2acb001bb346dd6ea11428f503fb1e9b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：局部最高概率为何不保证整句最高概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":"caption-decoding-later","formulas":[],"previousBlock":"b-318eba2140cce74164a6d0b1","nextBlock":"b-b7dda3daa921ecbce135ae19","prerequisites":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","relation":"same-reasoning"},{"id":"b-b7dda3daa921ecbce135ae19","url":"#b-b7dda3daa921ecbce135ae19","relation":"same-reasoning"},{"id":"b-fca31d6d6facb0c582ccf434","url":"#b-fca31d6d6facb0c582ccf434","relation":"same-reasoning"},{"id":"b-ef4ba79b1b52a3072a01a9fc","url":"#b-ef4ba79b1b52a3072a01a9fc","relation":"same-reasoning"}]},{"id":"b-b7dda3daa921ecbce135ae19","type":"p","scope":"caption-decoding-later","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-b7dda3daa921ecbce135ae19","content":"完整路径 AX、AY、BX、BY 的概率分别为 0.306、0.294、0.360、0.040。greedy 首先取 A，再取 X，得到 0.306；宽度 2 的 beam 首步保留 A/B，再展开并排序，能保留最终的 BX=0.36。这个可算小树解释局部选择与完整路径概率的差别。","sha256":"ca0781f38ff1958fb130064abb90ad8c2678b32c1241d77839edd7ae1ef6acf8","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：局部最高概率为何不保证整句最高概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":"caption-decoding-later","formulas":[],"previousBlock":"b-83a4df7935e400f89d3aa427","nextBlock":"b-fca31d6d6facb0c582ccf434","prerequisites":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","relation":"same-reasoning"},{"id":"b-83a4df7935e400f89d3aa427","url":"#b-83a4df7935e400f89d3aa427","relation":"same-reasoning"},{"id":"b-fca31d6d6facb0c582ccf434","url":"#b-fca31d6d6facb0c582ccf434","relation":"same-reasoning"},{"id":"b-ef4ba79b1b52a3072a01a9fc","url":"#b-ef4ba79b1b52a3072a01a9fc","relation":"same-reasoning"}]},{"id":"b-fca31d6d6facb0c582ccf434","type":"p","scope":"caption-decoding-later","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-fca31d6d6facb0c582ccf434","content":"有限 beam 仍可能在更大树里提前删掉最优前缀；更高模型概率也不保证更符合图像。如果模型把蓝杯误识成红杯，搜索更充分可能只是更坚定地输出错误描述。必须区分视觉建模错误、概率预测错误和搜索错误。","sha256":"2263210ab03d3652966597d08ee87e2d2abef3bd071157eb51a6ea67ca8520ea","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：局部最高概率为何不保证整句最高概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":"caption-decoding-later","formulas":[],"previousBlock":"b-b7dda3daa921ecbce135ae19","nextBlock":"b-ef4ba79b1b52a3072a01a9fc","prerequisites":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","relation":"same-reasoning"},{"id":"b-83a4df7935e400f89d3aa427","url":"#b-83a4df7935e400f89d3aa427","relation":"same-reasoning"},{"id":"b-b7dda3daa921ecbce135ae19","url":"#b-b7dda3daa921ecbce135ae19","relation":"same-reasoning"},{"id":"b-ef4ba79b1b52a3072a01a9fc","url":"#b-ef4ba79b1b52a3072a01a9fc","relation":"same-reasoning"}]},{"id":"b-ef4ba79b1b52a3072a01a9fc","type":"p","scope":"caption-decoding-later","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-ef4ba79b1b52a3072a01a9fc","content":"当前 LAB13 的实际模型中 greedy 与 beam=3 均为 100% 整句匹配，没有显示 beam 的经验优势。其热身代码采用另一棵归一化概率树，greedy=AZ、概率 0.3025，beam=BY、概率 0.36；两棵树各自定义自洽，不能把一棵树的首步概率与另一棵树的结果混用。","sha256":"92f197c441dd9a60ae9246f7eb4d102d3d469d13a8a8de0407e0c9c9aa009e0d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：局部最高概率为何不保证整句最高概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":"caption-decoding-later","formulas":[],"previousBlock":"b-fca31d6d6facb0c582ccf434","nextBlock":"b-178f6763e2fc8c22435ad239","prerequisites":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","relation":"same-reasoning"},{"id":"b-83a4df7935e400f89d3aa427","url":"#b-83a4df7935e400f89d3aa427","relation":"same-reasoning"},{"id":"b-b7dda3daa921ecbce135ae19","url":"#b-b7dda3daa921ecbce135ae19","relation":"same-reasoning"},{"id":"b-fca31d6d6facb0c582ccf434","url":"#b-fca31d6d6facb0c582ccf434","relation":"same-reasoning"}]},{"id":"b-178f6763e2fc8c22435ad239","type":"h3","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-178f6763e2fc8c22435ad239","content":"### 沿着图解追问","sha256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ef4ba79b1b52a3072a01a9fc","nextBlock":"b-022daeeec6a74aadc8c6f57f","prerequisites":[{"id":"b-318eba2140cce74164a6d0b1","url":"#b-318eba2140cce74164a6d0b1","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-022daeeec6a74aadc8c6f57f","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-022daeeec6a74aadc8c6f57f","content":"以下是备课用的模拟师生对话，不是课堂实录。","sha256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-178f6763e2fc8c22435ad239","nextBlock":"b-f703d0bd8fe547374bb378fd","prerequisites":[{"id":"b-178f6763e2fc8c22435ad239","url":"#b-178f6763e2fc8c22435ad239","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-f703d0bd8fe547374bb378fd","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-f703d0bd8fe547374bb378fd","content":"**学生：**给一张图多种描述，损失不是互相矛盾吗？","sha256":"6cb9ccaf588b2cc164fe529844e1e6102136545b9d2b6a2e359c3851f8b42743","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-022daeeec6a74aadc8c6f57f","nextBlock":"b-250d923240fb206c3daebcac","prerequisites":[{"id":"b-178f6763e2fc8c22435ad239","url":"#b-178f6763e2fc8c22435ad239","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-250d923240fb206c3daebcac","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-250d923240fb206c3daebcac","content":"**教师：**它们可以是同一条件下不同合理表达。我们学习条件分布，并由任务决定哪些信息必须覆盖；不是要求逐像素唯一地对应一句话。","sha256":"cb83be2919d657ca9b1de372dca7c1599a21baa73d45328e9c9fb394a115595d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-f703d0bd8fe547374bb378fd","nextBlock":"b-62c8e50d63ea377f16864ffe","prerequisites":[{"id":"b-178f6763e2fc8c22435ad239","url":"#b-178f6763e2fc8c22435ad239","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-62c8e50d63ea377f16864ffe","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-62c8e50d63ea377f16864ffe","content":"**学生：**训练时已经把前面的正确词给了模型，它会不会根本不看图？","sha256":"2856d764d26947c7c46b97ec0b11e760bbf74cdb47c3c344d3b51526d93b0ac6","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-250d923240fb206c3daebcac","nextBlock":"b-c643aa7bef222c64a4fe6a3a","prerequisites":[{"id":"b-178f6763e2fc8c22435ad239","url":"#b-178f6763e2fc8c22435ad239","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c643aa7bef222c64a4fe6a3a","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-c643aa7bef222c64a4fe6a3a","content":"**教师：**会有这种可能，所以还要从同一前缀自由生成，并交换或去掉图像。LAB13 保留的视觉消融就是在检查这一点。","sha256":"fc5702a73e802790c290d2df3ed9892ca3e2fdb119b5d282bd0d8437da0bd775","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-62c8e50d63ea377f16864ffe","nextBlock":"b-e7f0b4057cd5c2def8bd2c94","prerequisites":[{"id":"b-178f6763e2fc8c22435ad239","url":"#b-178f6763e2fc8c22435ad239","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-e7f0b4057cd5c2def8bd2c94","type":"h2","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-e7f0b4057cd5c2def8bd2c94","content":"## 沿这个问题继续","sha256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c643aa7bef222c64a4fe6a3a","nextBlock":"b-fd9592072f20916bf4a1c605","prerequisites":[{"id":"b-178f6763e2fc8c22435ad239","url":"#b-178f6763e2fc8c22435ad239","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-fd9592072f20916bf4a1c605","type":"p","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p51.html#b-fd9592072f20916bf4a1c605","content":"先回看本页画面并指出输入、条件和输出，再按“图像条件怎样进入一个可变长度答案 → 训练前缀、EOS 与 PAD：损失究竟算在哪些位置 → 已保存的条件生成实验：视觉输入确实改变了句子”阅读。相关实验以其已保存结果为起点，需要动手时再顺序执行 Notebook；","sha256":"d8c8ccccbd0b755c873b8e65c73b09831006e5dfdefc15ab254e6a0b7b49a5e0","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"generation-many-possible-results","title":"一个条件怎样容纳多种合理结果？"},"page":{"id":"caption-search-probability-extension","title":"选读：比较局部选择与完整路径概率","number":4},"sectionTitle":"选读：比较局部选择与完整路径概率","currentQuestion":"条件均值、合理样本与生成过程为什么是不同目标？","sourceSlides":["l3-r2-p51","l3-r2-p68"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e7f0b4057cd5c2def8bd2c94","nextBlock":null,"prerequisites":[{"id":"b-e7f0b4057cd5c2def8bd2c94","url":"#b-e7f0b4057cd5c2def8bd2c94","role":"section-title"},{"id":"b-93a7174a2a98e127794dcf5d","url":"#b-93a7174a2a98e127794dcf5d","role":"problem-context"}],"relatedBlocks":[]}]}