{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"在 z 上比较，为什么 h 仍会学习？","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html","markdown":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.md","pages":[{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","unit":"视图关系怎样成为学习目标，又会牺牲什么？","unitId":"i-view-objective","chapter":"I","number":1}],"aliases":[{"oldId":"b-1e023d98b3e6de11880176a7","targetId":"b-1e023d98b3e6de11880176a7","page":"reading-opening","operation":"move","beforeSHA256":"380b01b3335020a135fa773ef59ba41f22448b729b0ae1e8a4d9a9a029234110","afterSHA256":"380b01b3335020a135fa773ef59ba41f22448b729b0ae1e8a4d9a9a029234110","reason":"Same source body in authored semantic page"},{"oldId":"b-436e12e2b2396079c2fcc74b","targetId":"b-436e12e2b2396079c2fcc74b","page":"reading-opening","operation":"move","beforeSHA256":"6722cfa5d7120e481dca0b7392112a125d0cbcbf0710202dacecf6fd7fc4e4c4","afterSHA256":"6722cfa5d7120e481dca0b7392112a125d0cbcbf0710202dacecf6fd7fc4e4c4","reason":"Same source body in authored semantic page"},{"oldId":"b-d0d93886554fa590d4207b12","targetId":"b-d0d93886554fa590d4207b12","page":"reading-opening","operation":"move","beforeSHA256":"766530b00b724a8d074b82bb661c3a1f969e5c2d31d51e79c7b643acdaa729e3","afterSHA256":"766530b00b724a8d074b82bb661c3a1f969e5c2d31d51e79c7b643acdaa729e3","reason":"Same source body in authored semantic page"},{"oldId":"b-439d2c1f7185a7d816c92a87","targetId":"b-439d2c1f7185a7d816c92a87","page":"reading-opening","operation":"move","beforeSHA256":"b56d2e286f28991a29443b02caec3b3b2079a485b915d3698a9ebbfbb195b4b1","afterSHA256":"b56d2e286f28991a29443b02caec3b3b2079a485b915d3698a9ebbfbb195b4b1","reason":"Same source body in authored semantic page"},{"oldId":"b-fec2930fec27e3eb38863b7d","targetId":"b-fec2930fec27e3eb38863b7d","page":"reading-opening","operation":"move","beforeSHA256":"0196d0abdd7478e7e8f9463c5023498898e713c40e3b5065bc4883e2b222b1fb","afterSHA256":"0196d0abdd7478e7e8f9463c5023498898e713c40e3b5065bc4883e2b222b1fb","reason":"Same source body in authored semantic page"},{"oldId":"b-d50b5fce8a4d8c335406ae24","targetId":"b-d50b5fce8a4d8c335406ae24","page":"reading-opening","operation":"rewrite","beforeSHA256":"c0241b2b1fda34c90ce71024d36199ed917c2f47d9677ea9ad9d8300183b0cc5","afterSHA256":"bd320724a95547f3675d3929809a507c6cf680ed241cb04ed21c0662936d54d7","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-0d2ef78df16d8fe509d1a8a3","targetId":"b-0d2ef78df16d8fe509d1a8a3","page":"reading-opening","operation":"move","beforeSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","afterSHA256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-1e023d98b3e6de11880176a7","type":"h1","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-1e023d98b3e6de11880176a7","content":"# 在 z 上比较，为什么 h 仍会学习？","sha256":"380b01b3335020a135fa773ef59ba41f22448b729b0ae1e8a4d9a9a029234110","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-436e12e2b2396079c2fcc74b","prerequisites":[{"id":"b-436e12e2b2396079c2fcc74b","url":"#b-436e12e2b2396079c2fcc74b","role":"problem-context"}],"relatedBlocks":[{"id":"b-d50b5fce8a4d8c335406ae24","url":"#b-d50b5fce8a4d8c335406ae24","relation":"figure-and-explanation"}]},{"id":"b-436e12e2b2396079c2fcc74b","type":"p","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-436e12e2b2396079c2fcc74b","content":"解释训练空间与使用空间可以不同，同时明确投影头不截断梯度，也不保证保存一切。","sha256":"6722cfa5d7120e481dca0b7392112a125d0cbcbf0710202dacecf6fd7fc4e4c4","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-1e023d98b3e6de11880176a7","nextBlock":"b-d0d93886554fa590d4207b12","prerequisites":[{"id":"b-1e023d98b3e6de11880176a7","url":"#b-1e023d98b3e6de11880176a7","role":"section-title"}],"relatedBlocks":[{"id":"b-d50b5fce8a4d8c335406ae24","url":"#b-d50b5fce8a4d8c335406ae24","relation":"figure-and-explanation"}]},{"id":"b-d0d93886554fa590d4207b12","type":"p","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-d0d93886554fa590d4207b12","content":"训练路径是 $x\\to f_\\theta\\to h\\to g_\\phi\\to z\\to L$。投影头不是一道截断墙：链式法则中 $\\partial L/\\partial\\theta=(\\partial L/\\partial z)(\\partial z/\\partial h)(\\partial h/\\partial\\theta)$，梯度穿过g更新f。迁移时读取h，是选择另一个使用接口。","sha256":"766530b00b724a8d074b82bb661c3a1f969e5c2d31d51e79c7b643acdaa729e3","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"x\\to f_\\theta\\to h\\to g_\\phi\\to z\\to L","definitionContext":"在 z 上比较，为什么 h 仍会学习？"},{"latex":"\\partial L/\\partial\\theta=(\\partial L/\\partial z)(\\partial z/\\partial h)(\\partial h/\\partial\\theta)","definitionContext":"在 z 上比较，为什么 h 仍会学习？"}],"previousBlock":"b-436e12e2b2396079c2fcc74b","nextBlock":"b-439d2c1f7185a7d816c92a87","prerequisites":[{"id":"b-1e023d98b3e6de11880176a7","url":"#b-1e023d98b3e6de11880176a7","role":"section-title"},{"id":"b-436e12e2b2396079c2fcc74b","url":"#b-436e12e2b2396079c2fcc74b","role":"problem-context"}],"relatedBlocks":[{"id":"b-d50b5fce8a4d8c335406ae24","url":"#b-d50b5fce8a4d8c335406ae24","relation":"figure-and-explanation"}]},{"id":"b-439d2c1f7185a7d816c92a87","type":"p","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-439d2c1f7185a7d816c92a87","content":"考虑一个可行构造：$h=(u,c)$分别含形状与颜色，$g(h)=u$。两视图颜色不同但形状相同，那么z可以满足颜色不变性，而h仍有位置放颜色。这解释了投影头为什么可能缓和“训练要忽略”与“未来要保留”的冲突。","sha256":"b56d2e286f28991a29443b02caec3b3b2079a485b915d3698a9ebbfbb195b4b1","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"h=(u,c)","definitionContext":"在 z 上比较，为什么 h 仍会学习？"},{"latex":"g(h)=u","definitionContext":"在 z 上比较，为什么 h 仍会学习？"}],"previousBlock":"b-d0d93886554fa590d4207b12","nextBlock":"b-fec2930fec27e3eb38863b7d","prerequisites":[{"id":"b-1e023d98b3e6de11880176a7","url":"#b-1e023d98b3e6de11880176a7","role":"section-title"},{"id":"b-436e12e2b2396079c2fcc74b","url":"#b-436e12e2b2396079c2fcc74b","role":"problem-context"}],"relatedBlocks":[{"id":"b-d50b5fce8a4d8c335406ae24","url":"#b-d50b5fce8a4d8c335406ae24","relation":"figure-and-explanation"}]},{"id":"b-fec2930fec27e3eb38863b7d","type":"p","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-fec2930fec27e3eb38863b7d","content":"这个构造不保证网络一定学成这样。LAB 06的真实换色训练中，h的颜色5NN读出也接近机会水平（不等于所有读出都无法取得颜色）；有时z的形状读出反而优于h。正面理解是投影头提供了分工空间，实证问题则是训练后哪一层对哪项任务更有用。","sha256":"0196d0abdd7478e7e8f9463c5023498898e713c40e3b5065bc4883e2b222b1fb","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-439d2c1f7185a7d816c92a87","nextBlock":"b-d50b5fce8a4d8c335406ae24","prerequisites":[{"id":"b-1e023d98b3e6de11880176a7","url":"#b-1e023d98b3e6de11880176a7","role":"section-title"},{"id":"b-436e12e2b2396079c2fcc74b","url":"#b-436e12e2b2396079c2fcc74b","role":"problem-context"}],"relatedBlocks":[{"id":"b-d50b5fce8a4d8c335406ae24","url":"#b-d50b5fce8a4d8c335406ae24","relation":"figure-and-explanation"}]},{"id":"b-d50b5fce8a4d8c335406ae24","type":"figure","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-d50b5fce8a4d8c335406ae24","content":"图内标签与数值：P12 学习表示机制图，状态 1；预训练：损失作用在 z，梯度经过 f 与 g；视觉编码器 f；h；投影头 g；z；与对应视图 z⁺ 比较；损失 → g → f；两视图共享 f 与 g 的参数；右侧画出其中一条路径。\n\n损失在z计算，梯度经过投影头回到编码器。 查看对应图解状态\n\n- 预训练：损失作用在 z，梯度经过 f 与 g\n- 视觉编码器 f\n- h\n- 投影头 g\n- z\n- 与对应视图 z⁺ 比较\n- 损失 → g → f\n- 两视图共享 f 与 g 的参数；右侧画出其中一条路径。损失在z计算，梯度经过投影头回到编码器。 [查看对应图解状态](/#/l3-r2-p12/0)[查看完整图解](/#/l3-r2-p12/0)","sha256":"bd320724a95547f3675d3929809a507c6cf680ed241cb04ed21c0662936d54d7","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"损失在z计算，梯度经过投影头回到编码器。 查看对应图解状态","labels":["预训练：损失作用在 z，梯度经过 f 与 g","视觉编码器 f","h","投影头 g","z","与对应视图 z⁺ 比较","损失 → g → f","两视图共享 f 与 g 的参数；右侧画出其中一条路径。"],"description":"损失在z计算，梯度经过投影头回到编码器。 查看对应图解状态预训练：损失作用在 z，梯度经过 f 与 g视觉编码器 fh投影头 gz与对应视图 z⁺ 比较损失 → g → f两视图共享 f 与 g 的参数；右侧画出其中一条路径。","evidenceIdentity":"损失在z计算，梯度经过投影头回到编码器。 查看对应图解状态","images":[{"src":"/assets/display/9a66faeae8dd2c354bc56482.webp","alt":""}]},"previousBlock":"b-fec2930fec27e3eb38863b7d","nextBlock":"b-0d2ef78df16d8fe509d1a8a3","prerequisites":[{"id":"b-1e023d98b3e6de11880176a7","url":"#b-1e023d98b3e6de11880176a7","role":"section-title"},{"id":"b-436e12e2b2396079c2fcc74b","url":"#b-436e12e2b2396079c2fcc74b","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0d2ef78df16d8fe509d1a8a3","type":"h2","scope":"projection-head-gradient-and-retention","url":"https://codingai-lec03.pages.dev/course/blocks/projection-head-gradient-and-retention.html#b-0d2ef78df16d8fe509d1a8a3","content":"## 继续阅读与实验","sha256":"2b9ccef7e1220e2250cceff1abd8bdd736b42f8da5a223f26b029f44b962e807","chapter":{"id":"I","title":"学习信号与表示"},"unit":{"id":"i-view-objective","title":"视图关系怎样成为学习目标，又会牺牲什么？"},"page":{"id":"reading-opening","title":"在 z 上比较，为什么 h 仍会学习？","number":1},"sectionTitle":"在 z 上比较，为什么 h 仍会学习？","currentQuestion":"在 z 上比较，为什么 h 仍会学习？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d50b5fce8a4d8c335406ae24","nextBlock":null,"prerequisites":[{"id":"b-1e023d98b3e6de11880176a7","url":"#b-1e023d98b3e6de11880176a7","role":"section-title"},{"id":"b-436e12e2b2396079c2fcc74b","url":"#b-436e12e2b2396079c2fcc74b","role":"problem-context"}],"relatedBlocks":[{"id":"b-d50b5fce8a4d8c335406ae24","url":"#b-d50b5fce8a4d8c335406ae24","relation":"figure-and-explanation"}]}]}