{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"连接层与任务使用方式，都需要训练","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html","markdown":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.md","pages":[{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","unit":"可复用视觉与语言组件怎样学会共同回答？","unitId":"llava-interface-and-training","chapter":"III","number":1},{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","unit":"可复用视觉与语言组件怎样学会共同回答？","unitId":"llava-interface-and-training","chapter":"III","number":2}],"aliases":[{"oldId":"explanation","targetId":"llava-trainable-parameters","operation":"split-explanation-range"},{"oldId":"b-06728bc01e0f66302eb6d861","targetId":"b-06728bc01e0f66302eb6d861","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","afterSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","reason":"Same source body in authored semantic page"},{"oldId":"b-41c6ace095f7dcf70d32d2c7","targetId":"b-41c6ace095f7dcf70d32d2c7","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"a2a68083c3f59f82655a3722844588a3dc47cdbd24610f256b1082467c73b8f4","afterSHA256":"a2a68083c3f59f82655a3722844588a3dc47cdbd24610f256b1082467c73b8f4","reason":"Same source body in authored semantic page"},{"oldId":"b-f14bca485ec23546ba8f34db","targetId":"b-f14bca485ec23546ba8f34db","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"8a6a2a5319b9b8ab3a47b691fdbcd9ed861d802806a729e54755137d694c7761","afterSHA256":"8a6a2a5319b9b8ab3a47b691fdbcd9ed861d802806a729e54755137d694c7761","reason":"Same source body in authored semantic page"},{"oldId":"b-122baccc47eb5b39eed68349","targetId":"b-122baccc47eb5b39eed68349","page":"llava-trainable-parameters","operation":"rewrite","beforeSHA256":"c7b38910da76a7cf8c9d7bd04777168e7c165eb670b1ae24be020079c49b8f6c","afterSHA256":"2fd9c33581cc21eeca854cc24bc431133879e0b261c970974762a91dd4201ac4","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-ae1f9a9732f069b5a4dcb839","targetId":"b-ae1f9a9732f069b5a4dcb839","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","afterSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","reason":"Same source body in authored semantic page"},{"oldId":"b-2eeb39e800db8b8068c1724a","targetId":"b-2eeb39e800db8b8068c1724a","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"a2bb57764ec810701cb12af9c9094d953862415647a990469fcd3c99f9a91aac","afterSHA256":"a2bb57764ec810701cb12af9c9094d953862415647a990469fcd3c99f9a91aac","reason":"Same source body in authored semantic page"},{"oldId":"b-26408b0dbdc32e8d3e72242f","targetId":"b-26408b0dbdc32e8d3e72242f","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"a1b1181660cc8639edcc55ac705cb85a3f633a671c395e7839a55e0e3b2d13ce","afterSHA256":"a1b1181660cc8639edcc55ac705cb85a3f633a671c395e7839a55e0e3b2d13ce","reason":"Same source body in authored semantic page"},{"oldId":"b-edd7a784e922d874068b06a6","targetId":"b-edd7a784e922d874068b06a6","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"87a6785a721b7c4f15e50082fc0085dd997913f6a03a73a379a7a1107ed64174","afterSHA256":"87a6785a721b7c4f15e50082fc0085dd997913f6a03a73a379a7a1107ed64174","reason":"Same source body in authored semantic page"},{"oldId":"b-43a9f1807d363f7b29aecf6b","targetId":"b-43a9f1807d363f7b29aecf6b","page":"llava-trainable-parameters","operation":"rewrite","beforeSHA256":"30ffc21dc1b8b08cc65212c19d6afe1ccef2565d0b8013deb366b63f58f3f910","afterSHA256":"d29e0eaabd68eea2110a49c8c7a9e29cf3ecc4cec131d6744dac2d33dc456091","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-b9b48661b93a8efbe6de92ad","targetId":"b-b9b48661b93a8efbe6de92ad","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"5427ca1e3160be948135875e1b7ca9da3169818b912f9d963e8754786d7157a5","afterSHA256":"5427ca1e3160be948135875e1b7ca9da3169818b912f9d963e8754786d7157a5","reason":"Same source body in authored semantic page"},{"oldId":"b-073349518a66d0fd6ed5388c","targetId":"b-073349518a66d0fd6ed5388c","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"648327083320ae884440e22820ac2d90fa5606a4e6a8401b51445aeab27abd9f","afterSHA256":"648327083320ae884440e22820ac2d90fa5606a4e6a8401b51445aeab27abd9f","reason":"Same source body in authored semantic page"},{"oldId":"b-bbb80aea4f365a8dfff496b7","targetId":"b-bbb80aea4f365a8dfff496b7","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"fb8950c5997f55cc4a6d23292ef2faaae6ca0ceed6ddb10b3e2a54c54d6d9195","afterSHA256":"fb8950c5997f55cc4a6d23292ef2faaae6ca0ceed6ddb10b3e2a54c54d6d9195","reason":"Same source body in authored semantic page"},{"oldId":"b-bac6e05bbfbc33b4620c8e3e","targetId":"b-bac6e05bbfbc33b4620c8e3e","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"acd2f2f89ecc40ed95210430f3e60a592a14c339500cd8812ed744827c2a5d2e","afterSHA256":"acd2f2f89ecc40ed95210430f3e60a592a14c339500cd8812ed744827c2a5d2e","reason":"Same source body in authored semantic page"},{"oldId":"b-2b12915315b896c77260923d","targetId":"b-2b12915315b896c77260923d","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"faa469fdc5f32e70743738e98b9c29e30252246e2cde54d1f22ae7f010d39e18","afterSHA256":"faa469fdc5f32e70743738e98b9c29e30252246e2cde54d1f22ae7f010d39e18","reason":"Same source body in authored semantic page"},{"oldId":"b-3d07628048ba5563c90d806d","targetId":"b-3d07628048ba5563c90d806d","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"8c072ba9129ab07cc572f400fb6120b9102cfb1ffb05dd6d34086f3c58685c69","afterSHA256":"8c072ba9129ab07cc572f400fb6120b9102cfb1ffb05dd6d34086f3c58685c69","reason":"Same source body in authored semantic page"},{"oldId":"b-616821e0b946e4997e400393","targetId":"b-616821e0b946e4997e400393","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"5ce68457c32497fb8b214720c6483ac70007055868fc405f902cb285d9a74aca","afterSHA256":"5ce68457c32497fb8b214720c6483ac70007055868fc405f902cb285d9a74aca","reason":"Same source body in authored semantic page"},{"oldId":"b-d7d3b08663c12590c869ea08","targetId":"b-d7d3b08663c12590c869ea08","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"71ff06e8d13f49bba81e5746bdf0fa50cb9aa7c838bababf4d824bf50f3a7d54","afterSHA256":"71ff06e8d13f49bba81e5746bdf0fa50cb9aa7c838bababf4d824bf50f3a7d54","reason":"Same source body in authored semantic page"},{"oldId":"b-e51bf1fb7500eed883723a2b","targetId":"b-e51bf1fb7500eed883723a2b","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"01817e932f2fbc7bd5f40ecc653816c72c705a58c2525e001d5cc949628aadc5","afterSHA256":"01817e932f2fbc7bd5f40ecc653816c72c705a58c2525e001d5cc949628aadc5","reason":"Same source body in authored semantic page"},{"oldId":"b-86f607c5985be473065f59d3","targetId":"b-86f607c5985be473065f59d3","page":"llava-trainable-parameters","operation":"move","beforeSHA256":"096cab59bf5dc4f44f46e88a10601f4f06ae36234ed5dd8fc445efddc4e68fb0","afterSHA256":"096cab59bf5dc4f44f46e88a10601f4f06ae36234ed5dd8fc445efddc4e68fb0","reason":"Same source body in authored semantic page"},{"oldId":"b-b01fea6a639c9689a6b642d3","targetId":"b-b01fea6a639c9689a6b642d3","page":"llava-instruction-facts","operation":"move","beforeSHA256":"11db1c55e29b7105f55b4c0800a7b5f6040299e8f8456a866acadbbeebf48bfc","afterSHA256":"11db1c55e29b7105f55b4c0800a7b5f6040299e8f8456a866acadbbeebf48bfc","reason":"Same source body in authored semantic page"},{"oldId":"b-16fd10e2dc929dba65cb8753","targetId":"b-16fd10e2dc929dba65cb8753","page":"llava-instruction-facts","operation":"move","beforeSHA256":"4ce8535bd05ed62fc249519a252f16d353f88d71908287e3503be3b94df1b3ce","afterSHA256":"4ce8535bd05ed62fc249519a252f16d353f88d71908287e3503be3b94df1b3ce","reason":"Same source body in authored semantic page"},{"oldId":"b-78669ab1047dcc0c77c10c4e","targetId":"b-78669ab1047dcc0c77c10c4e","page":"llava-instruction-facts","operation":"move","beforeSHA256":"7621634e9af518b6512f680ae33d2ae56bb958cfe8455e7351064837bd6ebecd","afterSHA256":"7621634e9af518b6512f680ae33d2ae56bb958cfe8455e7351064837bd6ebecd","reason":"Same source body in authored semantic page"},{"oldId":"b-4bce398d4dabdf8f9186f879","targetId":"b-4bce398d4dabdf8f9186f879","page":"llava-instruction-facts","operation":"move","beforeSHA256":"89177feb914484f1328ffec9fc598ecd16f0720fe4a1e54e98cf3d537ad64582","afterSHA256":"89177feb914484f1328ffec9fc598ecd16f0720fe4a1e54e98cf3d537ad64582","reason":"Same source body in authored semantic page"},{"oldId":"b-c75ea0d1334530a4eb11daf3","targetId":"b-c75ea0d1334530a4eb11daf3","page":"llava-instruction-facts","operation":"move","beforeSHA256":"d8b80895551b3f9ccca4bc88b4c6aff4b122e58ce7c07d080abad2159d251c8d","afterSHA256":"d8b80895551b3f9ccca4bc88b4c6aff4b122e58ce7c07d080abad2159d251c8d","reason":"Same source body in authored semantic page"},{"oldId":"b-b4b2f6914daeea255b7987fd","targetId":"b-b4b2f6914daeea255b7987fd","page":"llava-instruction-facts","operation":"move","beforeSHA256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","afterSHA256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","reason":"Same source body in authored semantic page"},{"oldId":"b-8609bb68fcd2ba87732345b2","targetId":"b-8609bb68fcd2ba87732345b2","page":"llava-instruction-facts","operation":"move","beforeSHA256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","afterSHA256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","reason":"Same source body in authored semantic page"},{"oldId":"b-3ccfb1849579008c2e1c1130","targetId":"b-3ccfb1849579008c2e1c1130","page":"llava-instruction-facts","operation":"move","beforeSHA256":"fa5a7de56f57ab6501fa4a4cf7186f732d045f28dd516b9bd54e725d21c2b943","afterSHA256":"fa5a7de56f57ab6501fa4a4cf7186f732d045f28dd516b9bd54e725d21c2b943","reason":"Same source body in authored semantic page"},{"oldId":"b-352ff5acf3bf51e5b10d278c","targetId":"b-352ff5acf3bf51e5b10d278c","page":"llava-instruction-facts","operation":"move","beforeSHA256":"5fa14c1ef7101ae0b8cbe32ac344db2388e350a7692e521e0b0a5261106bb4e5","afterSHA256":"5fa14c1ef7101ae0b8cbe32ac344db2388e350a7692e521e0b0a5261106bb4e5","reason":"Same source body in authored semantic page"},{"oldId":"b-67ecf708bb626cadd7f454d1","targetId":"b-67ecf708bb626cadd7f454d1","page":"llava-instruction-facts","operation":"move","beforeSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","afterSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","reason":"Same source body in authored semantic page"},{"oldId":"b-03c382bb2721dd61ddec83b8","targetId":"b-03c382bb2721dd61ddec83b8","page":"llava-instruction-facts","operation":"move","beforeSHA256":"acd68459e4c7b9b50e941259100ad75faed53a6bd2ab07d73d52ca99f407a22b","afterSHA256":"acd68459e4c7b9b50e941259100ad75faed53a6bd2ab07d73d52ca99f407a22b","reason":"Same source body in authored semantic page"},{"oldId":"llava-trainable-parameters-title","targetId":"b-3dedcd04af64dee261654c9e","operation":"retain-source-identity"},{"oldId":"llava-instruction-facts-title","targetId":"b-d3096601c55514a0d6f3260c","operation":"retain-source-identity"},{"oldId":"llava-instruction-facts-bridge","targetId":"b-0d73a364deb5ecadaa0b230f","operation":"retain-source-identity"}],"blocks":[{"id":"b-3dedcd04af64dee261654c9e","type":"h2","scope":"llava-trainable-parameters","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-3dedcd04af64dee261654c9e","content":"## 保持模块不变，对照两阶段的更新范围","sha256":"43af51ad08af4d436577e7923163b1b745f26a364a2571238b855b46075a0898","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-06728bc01e0f66302eb6d861","prerequisites":[{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-06728bc01e0f66302eb6d861","type":"h2","scope":"explanation","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-06728bc01e0f66302eb6d861","content":"## 解释、算例与证据","sha256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3dedcd04af64dee261654c9e","nextBlock":"b-41c6ace095f7dcf70d32d2c7","prerequisites":[{"id":"b-3dedcd04af64dee261654c9e","url":"#b-3dedcd04af64dee261654c9e","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-41c6ace095f7dcf70d32d2c7","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-41c6ace095f7dcf70d32d2c7","content":"# P65 · 连接层与任务使用方式，都需要训练","sha256":"a2a68083c3f59f82655a3722844588a3dc47cdbd24610f256b1082467c73b8f4","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-06728bc01e0f66302eb6d861","nextBlock":"b-f14bca485ec23546ba8f34db","prerequisites":[{"id":"b-06728bc01e0f66302eb6d861","url":"#b-06728bc01e0f66302eb6d861","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-f14bca485ec23546ba8f34db","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-f14bca485ec23546ba8f34db","content":"本页两个状态只改变训练范围与数据任务：先冻结两大模型训练投影，再冻结视觉、更新投影和语言模型。把每个模块是否更新说清，避免用“端到端”概括掉细节。再读指令数据的来源，训练目标的视觉事实必须能追溯到相应输入。","sha256":"8a6a2a5319b9b8ab3a47b691fdbcd9ed861d802806a729e54755137d694c7761","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-41c6ace095f7dcf70d32d2c7","nextBlock":"b-122baccc47eb5b39eed68349","prerequisites":[{"id":"b-41c6ace095f7dcf70d32d2c7","url":"#b-41c6ace095f7dcf70d32d2c7","role":"section-title"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-122baccc47eb5b39eed68349","type":"figure","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-122baccc47eb5b39eed68349","content":"图内标签与数值：阶段二 · 图像指令训练任务使用；视觉编码器；冻结；保持参数；投影 W；可训练；更新参数；语言模型；可训练；更新参数；输入：图像 + 视觉指令；目标：指令对应的回答；可训练集合 { W, φ }；两个阶段都冻结视觉编码器；任务训练改变如何使用其特征。；限定原始 LLaVA §4.2。φ 为语言模型参数；不概括所有后续多模态模型的训练方案。\n\n原始LLaVA的两阶段图：阶段一只更新投影；阶段二更新投影与语言模型；两个阶段的视觉编码器均冻结。\n\n- 阶段二 · 图像指令训练任务使用\n- 视觉编码器\n- 冻结\n- 保持参数\n- 投影 W\n- 可训练\n- 更新参数\n- 语言模型\n- 可训练\n- 更新参数\n- 输入：图像 + 视觉指令\n- 目标：指令对应的回答\n- 可训练集合 { W, φ }\n- 两个阶段都冻结视觉编码器；任务训练改变如何使用其特征。\n- 限定原始 LLaVA §4.2。φ 为语言模型参数；不概括所有后续多模态模型的训练方案。原始LLaVA的两阶段图：阶段一只更新投影；阶段二更新投影与语言模型；两个阶段的视觉编码器均冻结。 [查看对应图解状态](/#/l3-r2-p65/1)[查看完整图解](/course/figures/l3-r2-p65-1.html)","sha256":"2fd9c33581cc21eeca854cc24bc431133879e0b261c970974762a91dd4201ac4","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"原始LLaVA的两阶段图：阶段一只更新投影；阶段二更新投影与语言模型；两个阶段的视觉编码器均冻结。 查看对应图解状态","labels":["阶段二 · 图像指令训练任务使用","视觉编码器","冻结","保持参数","投影 W","可训练","更新参数","语言模型","可训练","更新参数","输入：图像 + 视觉指令","目标：指令对应的回答","可训练集合 { W, φ }","两个阶段都冻结视觉编码器；任务训练改变如何使用其特征。","限定原始 LLaVA §4.2。φ 为语言模型参数；不概括所有后续多模态模型的训练方案。"],"description":"原始LLaVA的两阶段图：阶段一只更新投影；阶段二更新投影与语言模型；两个阶段的视觉编码器均冻结。阶段二 · 图像指令训练任务使用视觉编码器冻结保持参数投影 W可训练更新参数语言模型可训练更新参数输入：图像 + 视觉指令目标：指令对应的回答可训练集合 { W, φ }两个阶段都冻结视觉编码器；任务训练改变如何使用其特征。限定原始 LLaVA §4.2。φ 为语言模型参数；不概括所有后续多模态模型的训练方案。","evidenceIdentity":"原始LLaVA的两阶段图：阶段一只更新投影；阶段二更新投影与语言模型；两个阶段的视觉编码器均冻结。 查看对应图解状态","images":[]},"previousBlock":"b-f14bca485ec23546ba8f34db","nextBlock":"b-ae1f9a9732f069b5a4dcb839","prerequisites":[{"id":"b-41c6ace095f7dcf70d32d2c7","url":"#b-41c6ace095f7dcf70d32d2c7","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ae1f9a9732f069b5a4dcb839","type":"h2","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-ae1f9a9732f069b5a4dcb839","content":"## 课堂讲述","sha256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-122baccc47eb5b39eed68349","nextBlock":"b-2eeb39e800db8b8068c1724a","prerequisites":[{"id":"b-41c6ace095f7dcf70d32d2c7","url":"#b-41c6ace095f7dcf70d32d2c7","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-2eeb39e800db8b8068c1724a","type":"p","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-2eeb39e800db8b8068c1724a","content":"原始 LLaVA 的第一阶段冻结视觉编码器与语言模型，只训练投影，用图文配对建立接口。随后保持视觉编码器冻结，继续训练投影与语言模型，使系统适应视觉指令。\n这区分了两个问题：视觉特征怎样接进去，以及接进去后怎样按任务使用。具体数据与训练阶段会影响能力，不能把它解释成一根连接线带来了全部理解。\n我们也要用前面的办法检查结果：同一问题换图片，相关答案是否改变；同一图片换问题，是否提取了不同信息。能连贯回答是一项能力，回答是否有可靠视觉依据是另一项检查。两者共同构成真正有用的多模态系统。","sha256":"a2bb57764ec810701cb12af9c9094d953862415647a990469fcd3c99f9a91aac","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ae1f9a9732f069b5a4dcb839","nextBlock":"b-26408b0dbdc32e8d3e72242f","prerequisites":[{"id":"b-ae1f9a9732f069b5a4dcb839","url":"#b-ae1f9a9732f069b5a4dcb839","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-26408b0dbdc32e8d3e72242f","type":"h3","scope":"llava-two-stage-freezing","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-26408b0dbdc32e8d3e72242f","content":"### 原始 LLaVA 两阶段：学习接口，再学习视觉指令","sha256":"a1b1181660cc8639edcc55ac705cb85a3f633a671c395e7839a55e0e3b2d13ce","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"原始 LLaVA 两阶段：学习接口，再学习视觉指令","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-two-stage-freezing","formulas":[],"previousBlock":"b-2eeb39e800db8b8068c1724a","nextBlock":"b-edd7a784e922d874068b06a6","prerequisites":[{"id":"b-ae1f9a9732f069b5a4dcb839","url":"#b-ae1f9a9732f069b5a4dcb839","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"},{"id":"b-edd7a784e922d874068b06a6","url":"#b-edd7a784e922d874068b06a6","relation":"same-reasoning"},{"id":"b-43a9f1807d363f7b29aecf6b","url":"#b-43a9f1807d363f7b29aecf6b","relation":"same-reasoning"},{"id":"b-b9b48661b93a8efbe6de92ad","url":"#b-b9b48661b93a8efbe6de92ad","relation":"same-reasoning"},{"id":"b-073349518a66d0fd6ed5388c","url":"#b-073349518a66d0fd6ed5388c","relation":"same-reasoning"},{"id":"b-bbb80aea4f365a8dfff496b7","url":"#b-bbb80aea4f365a8dfff496b7","relation":"same-reasoning"}]},{"id":"b-edd7a784e922d874068b06a6","type":"p","scope":"llava-two-stage-freezing","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-edd7a784e922d874068b06a6","content":"冻结参数表示训练时不更新这组权重，不表示其输出不参与后续计算。原始 LLaVA 的训练范围必须按模块读：","sha256":"87a6785a721b7c4f15e50082fc0085dd997913f6a03a73a379a7a1107ed64174","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"原始 LLaVA 两阶段：学习接口，再学习视觉指令","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-two-stage-freezing","formulas":[],"previousBlock":"b-26408b0dbdc32e8d3e72242f","nextBlock":"b-43a9f1807d363f7b29aecf6b","prerequisites":[{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"},{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","relation":"same-reasoning"},{"id":"b-43a9f1807d363f7b29aecf6b","url":"#b-43a9f1807d363f7b29aecf6b","relation":"same-reasoning"},{"id":"b-b9b48661b93a8efbe6de92ad","url":"#b-b9b48661b93a8efbe6de92ad","relation":"same-reasoning"},{"id":"b-073349518a66d0fd6ed5388c","url":"#b-073349518a66d0fd6ed5388c","relation":"same-reasoning"},{"id":"b-bbb80aea4f365a8dfff496b7","url":"#b-bbb80aea4f365a8dfff496b7","relation":"same-reasoning"}]},{"id":"b-43a9f1807d363f7b29aecf6b","type":"table","scope":"llava-two-stage-freezing","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-43a9f1807d363f7b29aecf6b","content":"| 阶段 | 视觉编码器 | 线性投影 | 语言模型 | 主要任务 |\n| --- | --- | --- | --- | --- |\n| 特征对齐 | 冻结 | 更新 | 冻结 | 用图文配对建立可用视觉接口 |\n| 视觉指令微调 | 仍冻结 | 继续更新 | 更新 | 根据图片和指令生成回答 |","sha256":"d29e0eaabd68eea2110a49c8c7a9e29cf3ecc4cec131d6744dac2d33dc456091","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"原始 LLaVA 两阶段：学习接口，再学习视觉指令","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-two-stage-freezing","formulas":[],"table":{"headers":["阶段","视觉编码器","线性投影","语言模型","主要任务"],"rows":[["阶段","视觉编码器","线性投影","语言模型","主要任务"],["特征对齐","冻结","更新","冻结","用图文配对建立可用视觉接口"],["视觉指令微调","仍冻结","继续更新","更新","根据图片和指令生成回答"]]},"previousBlock":"b-edd7a784e922d874068b06a6","nextBlock":"b-b9b48661b93a8efbe6de92ad","prerequisites":[{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"},{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","relation":"same-reasoning"},{"id":"b-edd7a784e922d874068b06a6","url":"#b-edd7a784e922d874068b06a6","relation":"same-reasoning"},{"id":"b-b9b48661b93a8efbe6de92ad","url":"#b-b9b48661b93a8efbe6de92ad","relation":"same-reasoning"},{"id":"b-073349518a66d0fd6ed5388c","url":"#b-073349518a66d0fd6ed5388c","relation":"same-reasoning"},{"id":"b-bbb80aea4f365a8dfff496b7","url":"#b-bbb80aea4f365a8dfff496b7","relation":"same-reasoning"}]},{"id":"b-b9b48661b93a8efbe6de92ad","type":"p","scope":"llava-two-stage-freezing","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-b9b48661b93a8efbe6de92ad","content":"第一阶段让已有视觉表示进入语言接口，第二阶段进一步学习如何在不同问法下使用这些信息。两阶段的视觉参数都冻结；论文第二阶段标题虽含 End-to-End，不能据此把视觉分支也画成更新。","sha256":"5427ca1e3160be948135875e1b7ca9da3169818b912f9d963e8754786d7157a5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"原始 LLaVA 两阶段：学习接口，再学习视觉指令","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-two-stage-freezing","formulas":[],"previousBlock":"b-43a9f1807d363f7b29aecf6b","nextBlock":"b-073349518a66d0fd6ed5388c","prerequisites":[{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"},{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","relation":"same-reasoning"},{"id":"b-edd7a784e922d874068b06a6","url":"#b-edd7a784e922d874068b06a6","relation":"same-reasoning"},{"id":"b-43a9f1807d363f7b29aecf6b","url":"#b-43a9f1807d363f7b29aecf6b","relation":"same-reasoning"},{"id":"b-073349518a66d0fd6ed5388c","url":"#b-073349518a66d0fd6ed5388c","relation":"same-reasoning"},{"id":"b-bbb80aea4f365a8dfff496b7","url":"#b-bbb80aea4f365a8dfff496b7","relation":"same-reasoning"}]},{"id":"b-073349518a66d0fd6ed5388c","type":"p","scope":"llava-two-stage-freezing","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-073349518a66d0fd6ed5388c","content":"训练目标仍是条件下预测后续回答，只在应由助手生成的回答与停止位置计入目标。问题和图像属于条件，不是让模型原样抄回的答案。若只记录“用了指令数据”而不记录更新范围，就无法判断能力来自视觉编码变化、连接适配还是语言模型的任务行为变化。","sha256":"648327083320ae884440e22820ac2d90fa5606a4e6a8401b51445aeab27abd9f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"原始 LLaVA 两阶段：学习接口，再学习视觉指令","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-two-stage-freezing","formulas":[],"previousBlock":"b-b9b48661b93a8efbe6de92ad","nextBlock":"b-bbb80aea4f365a8dfff496b7","prerequisites":[{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"},{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","relation":"same-reasoning"},{"id":"b-edd7a784e922d874068b06a6","url":"#b-edd7a784e922d874068b06a6","relation":"same-reasoning"},{"id":"b-43a9f1807d363f7b29aecf6b","url":"#b-43a9f1807d363f7b29aecf6b","relation":"same-reasoning"},{"id":"b-b9b48661b93a8efbe6de92ad","url":"#b-b9b48661b93a8efbe6de92ad","relation":"same-reasoning"},{"id":"b-bbb80aea4f365a8dfff496b7","url":"#b-bbb80aea4f365a8dfff496b7","relation":"same-reasoning"}]},{"id":"b-bbb80aea4f365a8dfff496b7","type":"p","scope":"llava-two-stage-freezing","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-bbb80aea4f365a8dfff496b7","content":"这与第一部分冻结读出/微调的逻辑直接相连：不更新主干仍可以学习更有效的下游用法。其他多模态方案可以选择解冻视觉，这是一项新的训练设计，不能倒写成原始 LLaVA。","sha256":"fb8950c5997f55cc4a6d23292ef2faaae6ca0ceed6ddb10b3e2a54c54d6d9195","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"原始 LLaVA 两阶段：学习接口，再学习视觉指令","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-two-stage-freezing","formulas":[],"previousBlock":"b-073349518a66d0fd6ed5388c","nextBlock":"b-6e02dcefb5dbd4d10974bb63","prerequisites":[{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"},{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","relation":"same-reasoning"},{"id":"b-edd7a784e922d874068b06a6","url":"#b-edd7a784e922d874068b06a6","relation":"same-reasoning"},{"id":"b-43a9f1807d363f7b29aecf6b","url":"#b-43a9f1807d363f7b29aecf6b","relation":"same-reasoning"},{"id":"b-b9b48661b93a8efbe6de92ad","url":"#b-b9b48661b93a8efbe6de92ad","relation":"same-reasoning"},{"id":"b-073349518a66d0fd6ed5388c","url":"#b-073349518a66d0fd6ed5388c","relation":"same-reasoning"}]},{"id":"b-6e02dcefb5dbd4d10974bb63","type":"h4","scope":"llava-trainable-parameters","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-6e02dcefb5dbd4d10974bb63","content":"#### 图示条件与公式约定","sha256":"ef7f915c1ae63ed67bd23fecf050249273ba0efa8965d2c8d1c1c6b2d3178122","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-bbb80aea4f365a8dfff496b7","nextBlock":"b-bac6e05bbfbc33b4620c8e3e","prerequisites":[{"id":"b-26408b0dbdc32e8d3e72242f","url":"#b-26408b0dbdc32e8d3e72242f","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-bac6e05bbfbc33b4620c8e3e","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-bac6e05bbfbc33b4620c8e3e","content":"按原论文 §4.2：阶段一视觉编码器和语言模型冻结，只更新投影 W；阶段二继续冻结视觉编码器，更新投影 W 与语言模型参数 φ。没有把 End-to-End 标题误解为视觉也解冻。","sha256":"acd2f2f89ecc40ed95210430f3e60a592a14c339500cd8812ed744827c2a5d2e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-6e02dcefb5dbd4d10974bb63","nextBlock":"b-67089d029cb5b4e58ab471f4","prerequisites":[{"id":"b-6e02dcefb5dbd4d10974bb63","url":"#b-6e02dcefb5dbd4d10974bb63","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-67089d029cb5b4e58ab471f4","type":"h4","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-67089d029cb5b4e58ab471f4","content":"#### 这条线索的展开与后续阅读","sha256":"48b8229f7559b36ed4d5628416e3991e5c73293fbce43374db67386c8736f3bc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-bac6e05bbfbc33b4620c8e3e","nextBlock":"b-2b12915315b896c77260923d","prerequisites":[{"id":"b-6e02dcefb5dbd4d10974bb63","url":"#b-6e02dcefb5dbd4d10974bb63","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-2b12915315b896c77260923d","type":"ul","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-2b12915315b896c77260923d","content":"- [第一阶段：先让视觉接口接得上](/supplement.html?return=l3-r2-p65/0#/l3-llava-stage1/0) 本页说明\n- [第二阶段：让回答适配视觉指令](/supplement.html?return=l3-r2-p65/0#/l3-llava-stage2/0) 本页说明\n- [视觉指令材料从哪里来？](/supplement.html?return=l3-r2-p65/0#/l3-instruction-data/0) L03补充阅读\n- [视觉接入语言模型不是连一根线｜核心改动怎样工作](/supplement.html?return=l3-r2-p65/0#/l3-multimodal-interface-mechanism/0) L03补充阅读\n- [视觉接入语言模型不是连一根线｜什么结论还不能推出](/supplement.html?return=l3-r2-p65/0#/l3-multimodal-interface-boundary/0) L03补充阅读","sha256":"faa469fdc5f32e70743738e98b9c29e30252246e2cde54d1f22ae7f010d39e18","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-67089d029cb5b4e58ab471f4","nextBlock":"b-3d07628048ba5563c90d806d","prerequisites":[{"id":"b-67089d029cb5b4e58ab471f4","url":"#b-67089d029cb5b4e58ab471f4","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-3d07628048ba5563c90d806d","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-3d07628048ba5563c90d806d","content":"- [第一阶段：先让视觉接口接得上](/supplement.html?return=l3-r2-p65/0#/l3-llava-stage1/0) 本页说明","sha256":"8c072ba9129ab07cc572f400fb6120b9102cfb1ffb05dd6d34086f3c58685c69","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2b12915315b896c77260923d","nextBlock":"b-616821e0b946e4997e400393","prerequisites":[{"id":"b-67089d029cb5b4e58ab471f4","url":"#b-67089d029cb5b4e58ab471f4","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-616821e0b946e4997e400393","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-616821e0b946e4997e400393","content":"- [第二阶段：让回答适配视觉指令](/supplement.html?return=l3-r2-p65/0#/l3-llava-stage2/0) 本页说明","sha256":"5ce68457c32497fb8b214720c6483ac70007055868fc405f902cb285d9a74aca","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3d07628048ba5563c90d806d","nextBlock":"b-d7d3b08663c12590c869ea08","prerequisites":[{"id":"b-67089d029cb5b4e58ab471f4","url":"#b-67089d029cb5b4e58ab471f4","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-d7d3b08663c12590c869ea08","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-d7d3b08663c12590c869ea08","content":"- [视觉指令材料从哪里来？](/supplement.html?return=l3-r2-p65/0#/l3-instruction-data/0) L03补充阅读","sha256":"71ff06e8d13f49bba81e5746bdf0fa50cb9aa7c838bababf4d824bf50f3a7d54","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-616821e0b946e4997e400393","nextBlock":"b-e51bf1fb7500eed883723a2b","prerequisites":[{"id":"b-67089d029cb5b4e58ab471f4","url":"#b-67089d029cb5b4e58ab471f4","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-e51bf1fb7500eed883723a2b","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-e51bf1fb7500eed883723a2b","content":"- [视觉接入语言模型不是连一根线｜核心改动怎样工作](/supplement.html?return=l3-r2-p65/0#/l3-multimodal-interface-mechanism/0) L03补充阅读","sha256":"01817e932f2fbc7bd5f40ecc653816c72c705a58c2525e001d5cc949628aadc5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d7d3b08663c12590c869ea08","nextBlock":"b-86f607c5985be473065f59d3","prerequisites":[{"id":"b-67089d029cb5b4e58ab471f4","url":"#b-67089d029cb5b4e58ab471f4","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-86f607c5985be473065f59d3","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-86f607c5985be473065f59d3","content":"- [视觉接入语言模型不是连一根线｜什么结论还不能推出](/supplement.html?return=l3-r2-p65/0#/l3-multimodal-interface-boundary/0) L03补充阅读","sha256":"096cab59bf5dc4f44f46e88a10601f4f06ae36234ed5dd8fc445efddc4e68fb0","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-trainable-parameters","title":"保持模块不变，对照两阶段的更新范围","number":1},"sectionTitle":"保持模块不变，对照两阶段的更新范围","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e51bf1fb7500eed883723a2b","nextBlock":null,"prerequisites":[{"id":"b-67089d029cb5b4e58ab471f4","url":"#b-67089d029cb5b4e58ab471f4","role":"section-title"},{"id":"b-f14bca485ec23546ba8f34db","url":"#b-f14bca485ec23546ba8f34db","role":"problem-context"}],"relatedBlocks":[{"id":"b-122baccc47eb5b39eed68349","url":"#b-122baccc47eb5b39eed68349","relation":"figure-and-explanation"}]},{"id":"b-d3096601c55514a0d6f3260c","type":"h2","scope":"llava-instruction-facts","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-d3096601c55514a0d6f3260c","content":"## 沿指令数据追溯答案能依据的事实","sha256":"0ff2b1ea4a85f7fdaa312c7d9720c32406c8a01f118b10d100b9ae454dfc244d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-0d73a364deb5ecadaa0b230f","prerequisites":[{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0d73a364deb5ecadaa0b230f","type":"p","scope":"llava-instruction-facts","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-0d73a364deb5ecadaa0b230f","content":"参数更新范围已清楚；还要检查训练答案是否有足够的输入证据。","sha256":"7fb1eb58f8e341e146c262375abbf0045f63a10841ff15dc11e9364257804cdf","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d3096601c55514a0d6f3260c","nextBlock":"b-b01fea6a639c9689a6b642d3","prerequisites":[{"id":"b-d3096601c55514a0d6f3260c","url":"#b-d3096601c55514a0d6f3260c","role":"section-title"}],"relatedBlocks":[]},{"id":"b-b01fea6a639c9689a6b642d3","type":"h3","scope":"llava-instruction-source","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-b01fea6a639c9689a6b642d3","content":"### 视觉指令数据：问题和答案中的事实从哪里来","sha256":"11db1c55e29b7105f55b4c0800a7b5f6040299e8f8456a866acadbbeebf48bfc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"视觉指令数据：问题和答案中的事实从哪里来","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-instruction-source","formulas":[],"previousBlock":"b-0d73a364deb5ecadaa0b230f","nextBlock":"b-16fd10e2dc929dba65cb8753","prerequisites":[{"id":"b-d3096601c55514a0d6f3260c","url":"#b-d3096601c55514a0d6f3260c","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[{"id":"b-16fd10e2dc929dba65cb8753","url":"#b-16fd10e2dc929dba65cb8753","relation":"same-reasoning"},{"id":"b-78669ab1047dcc0c77c10c4e","url":"#b-78669ab1047dcc0c77c10c4e","relation":"same-reasoning"},{"id":"b-4bce398d4dabdf8f9186f879","url":"#b-4bce398d4dabdf8f9186f879","relation":"same-reasoning"},{"id":"b-c75ea0d1334530a4eb11daf3","url":"#b-c75ea0d1334530a4eb11daf3","relation":"same-reasoning"}]},{"id":"b-16fd10e2dc929dba65cb8753","type":"p","scope":"llava-instruction-source","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-16fd10e2dc929dba65cb8753","content":"原始 LLaVA 的一部分指令资料由仅接收文本的 GPT-4 协助制作。输入是已有 caption 和对象类别、框坐标等文字化描述；这条数据制作路径没有让 GPT-4 直接查看原图。随后训练 LLaVA 时，原图才作为视觉输入与这些指令配对。","sha256":"4ce8535bd05ed62fc249519a252f16d353f88d71908287e3503be3b94df1b3ce","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"视觉指令数据：问题和答案中的事实从哪里来","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-instruction-source","formulas":[],"previousBlock":"b-b01fea6a639c9689a6b642d3","nextBlock":"b-78669ab1047dcc0c77c10c4e","prerequisites":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","relation":"same-reasoning"},{"id":"b-78669ab1047dcc0c77c10c4e","url":"#b-78669ab1047dcc0c77c10c4e","relation":"same-reasoning"},{"id":"b-4bce398d4dabdf8f9186f879","url":"#b-4bce398d4dabdf8f9186f879","relation":"same-reasoning"},{"id":"b-c75ea0d1334530a4eb11daf3","url":"#b-c75ea0d1334530a4eb11daf3","relation":"same-reasoning"}]},{"id":"b-78669ab1047dcc0c77c10c4e","type":"p","scope":"llava-instruction-source","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-78669ab1047dcc0c77c10c4e","content":"把一份场景记录想成三层。caption 可能说“两人在球场”；对象框提供人物和球的位置；若记录没有谁的手接触球、哪一人持球，数据制作模型就缺乏可靠依据回答持球关系。把回答写得更长不能补上丢失事实。需要追加标注、改问题或保留不确定性。","sha256":"7621634e9af518b6512f680ae33d2ae56bb958cfe8455e7351064837bd6ebecd","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"视觉指令数据：问题和答案中的事实从哪里来","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-instruction-source","formulas":[],"previousBlock":"b-16fd10e2dc929dba65cb8753","nextBlock":"b-4bce398d4dabdf8f9186f879","prerequisites":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","relation":"same-reasoning"},{"id":"b-16fd10e2dc929dba65cb8753","url":"#b-16fd10e2dc929dba65cb8753","relation":"same-reasoning"},{"id":"b-4bce398d4dabdf8f9186f879","url":"#b-4bce398d4dabdf8f9186f879","relation":"same-reasoning"},{"id":"b-c75ea0d1334530a4eb11daf3","url":"#b-c75ea0d1334530a4eb11daf3","relation":"same-reasoning"}]},{"id":"b-4bce398d4dabdf8f9186f879","type":"p","scope":"llava-instruction-source","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-4bce398d4dabdf8f9186f879","content":"这种流程的积极价值是把已有图文资料转为更丰富的任务形式，让模型学习同图换问、描述与解释等交互。数据生产因此不只是在扩大样本数量，还在定义系统练习哪些条件使用方式。","sha256":"89177feb914484f1328ffec9fc598ecd16f0720fe4a1e54e98cf3d537ad64582","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"视觉指令数据：问题和答案中的事实从哪里来","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-instruction-source","formulas":[],"previousBlock":"b-78669ab1047dcc0c77c10c4e","nextBlock":"b-c75ea0d1334530a4eb11daf3","prerequisites":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","relation":"same-reasoning"},{"id":"b-16fd10e2dc929dba65cb8753","url":"#b-16fd10e2dc929dba65cb8753","relation":"same-reasoning"},{"id":"b-78669ab1047dcc0c77c10c4e","url":"#b-78669ab1047dcc0c77c10c4e","relation":"same-reasoning"},{"id":"b-c75ea0d1334530a4eb11daf3","url":"#b-c75ea0d1334530a4eb11daf3","relation":"same-reasoning"}]},{"id":"b-c75ea0d1334530a4eb11daf3","type":"p","scope":"llava-instruction-source","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-c75ea0d1334530a4eb11daf3","content":"学生可以对一张课程图自己做一次检查：只给对象名和框，哪些问题已能确定，哪些要看像素才知道？“左右各是谁”可能由框和类别支持；球面的小字、遮挡细节或动作接触通常需要更细证据。这与本讲开头“训练答案怎样产生”是同一个问题。","sha256":"d8b80895551b3f9ccca4bc88b4c6aff4b122e58ce7c07d080abad2159d251c8d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"视觉指令数据：问题和答案中的事实从哪里来","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":"llava-instruction-source","formulas":[],"previousBlock":"b-4bce398d4dabdf8f9186f879","nextBlock":"b-b4b2f6914daeea255b7987fd","prerequisites":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","relation":"same-reasoning"},{"id":"b-16fd10e2dc929dba65cb8753","url":"#b-16fd10e2dc929dba65cb8753","relation":"same-reasoning"},{"id":"b-78669ab1047dcc0c77c10c4e","url":"#b-78669ab1047dcc0c77c10c4e","relation":"same-reasoning"},{"id":"b-4bce398d4dabdf8f9186f879","url":"#b-4bce398d4dabdf8f9186f879","relation":"same-reasoning"}]},{"id":"b-b4b2f6914daeea255b7987fd","type":"h3","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-b4b2f6914daeea255b7987fd","content":"### 沿着图解追问","sha256":"f16be0424762a44344338696f06ef266c4571d15367ae2cbbd28fac98ac7e9cb","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c75ea0d1334530a4eb11daf3","nextBlock":"b-8609bb68fcd2ba87732345b2","prerequisites":[{"id":"b-b01fea6a639c9689a6b642d3","url":"#b-b01fea6a639c9689a6b642d3","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8609bb68fcd2ba87732345b2","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-8609bb68fcd2ba87732345b2","content":"以下是备课用的模拟师生对话，不是课堂实录。","sha256":"b25685f04cd1b885f3a5999e6a912460c7424021cdc5b2190584242086a82681","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b4b2f6914daeea255b7987fd","nextBlock":"b-3ccfb1849579008c2e1c1130","prerequisites":[{"id":"b-b4b2f6914daeea255b7987fd","url":"#b-b4b2f6914daeea255b7987fd","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3ccfb1849579008c2e1c1130","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-3ccfb1849579008c2e1c1130","content":"**学生：**视觉冻结是不是完全不能学习新视觉任务？","sha256":"fa5a7de56f57ab6501fa4a4cf7186f732d045f28dd516b9bd54e725d21c2b943","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8609bb68fcd2ba87732345b2","nextBlock":"b-352ff5acf3bf51e5b10d278c","prerequisites":[{"id":"b-b4b2f6914daeea255b7987fd","url":"#b-b4b2f6914daeea255b7987fd","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-352ff5acf3bf51e5b10d278c","type":"p","scope":"dialogue","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-352ff5acf3bf51e5b10d278c","content":"**教师：**视觉参数不变，但投影和语言模型可以学习新的读取与使用方式。若已有特征确实缺少细节，才需要考虑分辨率、表示或解冻等进一步改动。","sha256":"5fa14c1ef7101ae0b8cbe32ac344db2388e350a7692e521e0b0a5261106bb4e5","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3ccfb1849579008c2e1c1130","nextBlock":"b-67ecf708bb626cadd7f454d1","prerequisites":[{"id":"b-b4b2f6914daeea255b7987fd","url":"#b-b4b2f6914daeea255b7987fd","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-67ecf708bb626cadd7f454d1","type":"h2","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-67ecf708bb626cadd7f454d1","content":"## 沿这个问题继续","sha256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-352ff5acf3bf51e5b10d278c","nextBlock":"b-03c382bb2721dd61ddec83b8","prerequisites":[{"id":"b-b4b2f6914daeea255b7987fd","url":"#b-b4b2f6914daeea255b7987fd","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-03c382bb2721dd61ddec83b8","type":"p","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p65.html#b-03c382bb2721dd61ddec83b8","content":"对照两阶段的冻结范围与数据任务，再追溯指令数据中的问题、答案和图像事实。用同一组模块说明：训练接口与学习回答分别改变了什么。","sha256":"acd68459e4c7b9b50e941259100ad75faed53a6bd2ab07d73d52ca99f407a22b","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"llava-interface-and-training","title":"可复用视觉与语言组件怎样学会共同回答？"},"page":{"id":"llava-instruction-facts","title":"沿指令数据追溯答案能依据的事实","number":2},"sectionTitle":"沿指令数据追溯答案能依据的事实","currentQuestion":"接口、更新参数和指令事实各自需要怎样检查？","sourceSlides":["l3-r2-p65"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-67ecf708bb626cadd7f454d1","nextBlock":null,"prerequisites":[{"id":"b-67ecf708bb626cadd7f454d1","url":"#b-67ecf708bb626cadd7f454d1","role":"section-title"},{"id":"b-0d73a364deb5ecadaa0b230f","url":"#b-0d73a364deb5ecadaa0b230f","role":"problem-context"}],"relatedBlocks":[]}]}