{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"教师不一定是更大的模型，也不一定提供真值","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.md","pages":[{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","unit":"教师不一定是更大的模型，也不一定提供真值","unitId":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","chapter":"course","number":1}],"aliases":[{"oldId":"b-16cdf10e3925ad469a43f0d3","targetId":"b-16cdf10e3925ad469a43f0d3","page":"reading-opening","operation":"move","beforeSHA256":"44a74e10a676a4279d4180fb4458aee1016d4cdeba4b41a1d9e7798e3be61ed1","afterSHA256":"44a74e10a676a4279d4180fb4458aee1016d4cdeba4b41a1d9e7798e3be61ed1","reason":"Same source body in authored semantic page"},{"oldId":"b-df469e5506abdeada1e506cd","targetId":"b-df469e5506abdeada1e506cd","page":"reading-opening","operation":"move","beforeSHA256":"768df6aec63d3d44f01b7514e7e1d997d377605b27a7a7bbd2ce0cc35c6d6e48","afterSHA256":"768df6aec63d3d44f01b7514e7e1d997d377605b27a7a7bbd2ce0cc35c6d6e48","reason":"Same source body in authored semantic page"},{"oldId":"b-11f873e33e1fbb990aed5125","targetId":"b-11f873e33e1fbb990aed5125","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-ec15d543ec9eed6a8a948c42","targetId":"b-ec15d543ec9eed6a8a948c42","page":"reading-opening","operation":"move","beforeSHA256":"44af65f60193d6766e8cc6019f133e85a56a9751792313a8cd2e06a0256ec27f","afterSHA256":"44af65f60193d6766e8cc6019f133e85a56a9751792313a8cd2e06a0256ec27f","reason":"Same source body in authored semantic page"},{"oldId":"b-e53607aaea98e4b54b11f436","targetId":"b-e53607aaea98e4b54b11f436","page":"reading-opening","operation":"move","beforeSHA256":"efc07be0d238c97b2a0f2a3e1e141f27691e7d1aadb5b0bc2662d7d40218327f","afterSHA256":"efc07be0d238c97b2a0f2a3e1e141f27691e7d1aadb5b0bc2662d7d40218327f","reason":"Same source body in authored semantic page"},{"oldId":"b-d92245ec238dc8448d8926bc","targetId":"b-d92245ec238dc8448d8926bc","page":"reading-opening","operation":"move","beforeSHA256":"7a09424a9fcff03809af57ece9eea1d6c350b11893d6409472958f4ef32264b2","afterSHA256":"7a09424a9fcff03809af57ece9eea1d6c350b11893d6409472958f4ef32264b2","reason":"Same source body in authored semantic page"},{"oldId":"b-d437fe357ba4cc2d964e0391","targetId":"b-d437fe357ba4cc2d964e0391","page":"reading-opening","operation":"move","beforeSHA256":"e15af766bfdd95248a72eee449be07b21825db5fb82400539f177dbfbcf75e04","afterSHA256":"e15af766bfdd95248a72eee449be07b21825db5fb82400539f177dbfbcf75e04","reason":"Same source body in authored semantic page"},{"oldId":"b-ddf24d2230d3b6247868f667","targetId":"b-ddf24d2230d3b6247868f667","page":"reading-opening","operation":"move","beforeSHA256":"a4194bffb03d98e3ab6ee0e192440540c4b75b3bbddb7d347c41aed9223cd2d7","afterSHA256":"a4194bffb03d98e3ab6ee0e192440540c4b75b3bbddb7d347c41aed9223cd2d7","reason":"Same source body in authored semantic page"},{"oldId":"b-5c1e45d5b561983801e40bd0","targetId":"b-5c1e45d5b561983801e40bd0","page":"reading-opening","operation":"move","beforeSHA256":"0e85b26615f1bb8ce7feef1df33b086ae214edf48843560f3591ee71d79f0e2e","afterSHA256":"0e85b26615f1bb8ce7feef1df33b086ae214edf48843560f3591ee71d79f0e2e","reason":"Same source body in authored semantic page"},{"oldId":"b-2307962d4a40747844e798d7","targetId":"b-2307962d4a40747844e798d7","page":"reading-opening","operation":"move","beforeSHA256":"2ef2ec8a8fd967adffc562707501137a4fcc967a2af3e53e4d167b0e0f1aff51","afterSHA256":"2ef2ec8a8fd967adffc562707501137a4fcc967a2af3e53e4d167b0e0f1aff51","reason":"Same source body in authored semantic page"},{"oldId":"b-2969278a28f21414a92667dc","targetId":"b-2969278a28f21414a92667dc","page":"reading-opening","operation":"move","beforeSHA256":"4730b4cad436cd4b274c52e198e7548d06b30738b1d878fb35771460fd51ec8f","afterSHA256":"4730b4cad436cd4b274c52e198e7548d06b30738b1d878fb35771460fd51ec8f","reason":"Same source body in authored semantic page"},{"oldId":"b-bc5500b59c04052396feb1e7","targetId":"b-bc5500b59c04052396feb1e7","page":"reading-opening","operation":"move","beforeSHA256":"18d41b1e5728c89da6ca0c68d3702e0dd7c3d817a177e150d1c544abfb081458","afterSHA256":"18d41b1e5728c89da6ca0c68d3702e0dd7c3d817a177e150d1c544abfb081458","reason":"Same source body in authored semantic page"},{"oldId":"b-c00d9c4af0d4437b1974e41a","targetId":"b-c00d9c4af0d4437b1974e41a","page":"reading-opening","operation":"move","beforeSHA256":"9daa40692cb1e71c17ff59655c00da11e6e36068fd5ca5549da5016ecb2f031e","afterSHA256":"9daa40692cb1e71c17ff59655c00da11e6e36068fd5ca5549da5016ecb2f031e","reason":"Same source body in authored semantic page"},{"oldId":"b-50253e0da6b6945bddea1b79","targetId":"b-50253e0da6b6945bddea1b79","page":"reading-opening","operation":"move","beforeSHA256":"60c0021cf3e308ab09b33e2eaa8dcc837399ca1d7ea911e4c5988bde77692bd7","afterSHA256":"60c0021cf3e308ab09b33e2eaa8dcc837399ca1d7ea911e4c5988bde77692bd7","reason":"Same source body in authored semantic page"},{"oldId":"b-09745ff18aff64962f40883e","targetId":"b-09745ff18aff64962f40883e","page":"reading-opening","operation":"move","beforeSHA256":"75a4add903cc4a1e6fb297730fba5165423f8e20dbba8243cadb62f713b8be20","afterSHA256":"75a4add903cc4a1e6fb297730fba5165423f8e20dbba8243cadb62f713b8be20","reason":"Same source body in authored semantic page"},{"oldId":"b-9fcc8e72ee0f8dd25e4c584a","targetId":"b-9fcc8e72ee0f8dd25e4c584a","page":"reading-opening","operation":"move","beforeSHA256":"792d216843f4f01968151809602fa8c84963defa7cf462797140948c3f044a6b","afterSHA256":"792d216843f4f01968151809602fa8c84963defa7cf462797140948c3f044a6b","reason":"Same source body in authored semantic page"},{"oldId":"b-7ff5ba3b13efb7116ddb12f4","targetId":"b-7ff5ba3b13efb7116ddb12f4","page":"reading-opening","operation":"rewrite","beforeSHA256":"c21902cabd20ed4ed1f16ebb7e6e27c94c38ea438454af6ee6ffb747aabec209","afterSHA256":"5bea10eb69b97fe9ff1f3306fc82766b9958278943003d0ce821c7862c98df24","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-eba4d4299b15149ffb382cf1","targetId":"b-eba4d4299b15149ffb382cf1","page":"reading-opening","operation":"move","beforeSHA256":"ad1c9cdc8c931315cefa1f74029155573a31c600a5d190e736b1616f4743d31a","afterSHA256":"ad1c9cdc8c931315cefa1f74029155573a31c600a5d190e736b1616f4743d31a","reason":"Same source body in authored semantic page"},{"oldId":"block-1861e17c9d7c49f0970e890873be4ff6--教师不一定是更大的模型也不一定提供真值","targetId":"b-c00d9c4af0d4437b1974e41a","operation":"retain-source-identity"}],"blocks":[{"id":"b-16cdf10e3925ad469a43f0d3","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-16cdf10e3925ad469a43f0d3","content":"[CLS 06 · 表示的三个检查：目标、信息与新域](/course/notebooks/c0645a4db6e04188ab5b119e87c53bed.html)","sha256":"44a74e10a676a4279d4180fb4458aee1016d4cdeba4b41a1d9e7798e3be61ed1","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-df469e5506abdeada1e506cd","prerequisites":[],"relatedBlocks":[]},{"id":"b-df469e5506abdeada1e506cd","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-df469e5506abdeada1e506cd","content":"# 教师不一定是更大的模型，也不一定提供真值","sha256":"768df6aec63d3d44f01b7514e7e1d997d377605b27a7a7bbd2ce0cc35c6d6e48","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-16cdf10e3925ad469a43f0d3","nextBlock":"b-11f873e33e1fbb990aed5125","prerequisites":[{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-11f873e33e1fbb990aed5125","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-11f873e33e1fbb990aed5125","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-df469e5506abdeada1e506cd","nextBlock":"b-ec15d543ec9eed6a8a948c42","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ec15d543ec9eed6a8a948c42","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-ec15d543ec9eed6a8a948c42","content":"教师软目标、坍塌、任务信息与域变化的三个不同问题","sha256":"44af65f60193d6766e8cc6019f133e85a56a9751792313a8cd2e06a0256ec27f","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-11f873e33e1fbb990aed5125","nextBlock":"b-e53607aaea98e4b54b11f436","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-e53607aaea98e4b54b11f436","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-e53607aaea98e4b54b11f436","content":"**学习路径：** [表示怎样迁移到新任务](/course/study/readout-transfer.html)","sha256":"efc07be0d238c97b2a0f2a3e1e141f27691e7d1aadb5b0bc2662d7d40218327f","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ec15d543ec9eed6a8a948c42","nextBlock":"b-d92245ec238dc8448d8926bc","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d92245ec238dc8448d8926bc","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-d92245ec238dc8448d8926bc","content":"从[P23评价协议](/course/pages/l3-r2-p23.html)或[P24新任务](/course/pages/l3-r2-p24.html)带一个问题进入，不必顺序读完三条分支。","sha256":"7a09424a9fcff03809af57ece9eea1d6c350b11893d6409472958f4ef32264b2","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e53607aaea98e4b54b11f436","nextBlock":"b-d437fe357ba4cc2d964e0391","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d437fe357ba4cc2d964e0391","type":"ul","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-d437fe357ba4cc2d964e0391","content":"- **目标是否允许无用的解？** 看常量表示为何得到零一致性损失，再用[LAB 17](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/268a4ab11b464c64b065f9fc12b19cc7)区分固定教师拟合与独立EMA计算。\n\n- **线性读不出是否等于信息丢失？** 看[XOR四点图](/course/blocks/information-and-readable-form.html)，再用[LAB 18](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/33066318f5884adc8417068df0f40431)比较标签信息、身份信息和读出形式。\n\n- **换环境失效是否就是负迁移？** 看[环境变化与负迁移](/course/blocks/environment-shift-versus-negative-transfer.html)，再用[LAB 19](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/c20ef6a65d184331b1b52bf351077504)分别计算适配收益与相对scratch收益。","sha256":"e15af766bfdd95248a72eee449be07b21825db5fb82400539f177dbfbcf75e04","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d92245ec238dc8448d8926bc","nextBlock":"b-ddf24d2230d3b6247868f667","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ddf24d2230d3b6247868f667","type":"li","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-ddf24d2230d3b6247868f667","content":"- **目标是否允许无用的解？** 看常量表示为何得到零一致性损失，再用[LAB 17](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/268a4ab11b464c64b065f9fc12b19cc7)区分固定教师拟合与独立EMA计算。","sha256":"a4194bffb03d98e3ab6ee0e192440540c4b75b3bbddb7d347c41aed9223cd2d7","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d437fe357ba4cc2d964e0391","nextBlock":"b-5c1e45d5b561983801e40bd0","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-5c1e45d5b561983801e40bd0","type":"li","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-5c1e45d5b561983801e40bd0","content":"- **线性读不出是否等于信息丢失？** 看[XOR四点图](/course/blocks/information-and-readable-form.html)，再用[LAB 18](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/33066318f5884adc8417068df0f40431)比较标签信息、身份信息和读出形式。","sha256":"0e85b26615f1bb8ce7feef1df33b086ae214edf48843560f3591ee71d79f0e2e","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ddf24d2230d3b6247868f667","nextBlock":"b-2307962d4a40747844e798d7","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2307962d4a40747844e798d7","type":"li","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-2307962d4a40747844e798d7","content":"- **换环境失效是否就是负迁移？** 看[环境变化与负迁移](/course/blocks/environment-shift-versus-negative-transfer.html)，再用[LAB 19](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/c20ef6a65d184331b1b52bf351077504)分别计算适配收益与相对scratch收益。","sha256":"2ef2ec8a8fd967adffc562707501137a4fcc967a2af3e53e4d167b0e0f1aff51","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-5c1e45d5b561983801e40bd0","nextBlock":"b-2969278a28f21414a92667dc","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2969278a28f21414a92667dc","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-2969278a28f21414a92667dc","content":"留下一个可检验判断即可：指出你比较的是哪两个对象、固定了什么条件、结果排除了哪种解释。若要用真实图像检查冻结与微调，回到[LAB 01](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/4dd8605ed9904820919ea9e13d9e7d7c)/[LAB 02](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/4bd450d54870432db91c80a4df563b8e)。","sha256":"4730b4cad436cd4b274c52e198e7548d06b30738b1d878fb35771460fd51ec8f","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2307962d4a40747844e798d7","nextBlock":"b-bc5500b59c04052396feb1e7","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-bc5500b59c04052396feb1e7","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-bc5500b59c04052396feb1e7","content":"对比学习用负例阻止所有样本挤成一点。如果去掉负例，只让同图两视图接近，常量表示 $f(x)=c$ 对任意输入都能达到零距离。这个解完全满足“彼此一致”，却不能区分杯子和书。于是问题从“怎样产生训练目标”推进为：目标本身是否排除了我们不想要的解？本册把教师学习、表示信息和域变化连起来，是因为三者都要求分清**模型容易满足的条件**和**我们真正需要的能力**。","sha256":"18d41b1e5728c89da6ca0c68d3702e0dd7c3d817a177e150d1c544abfb081458","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"f(x)=c","definitionContext":"教师不一定是更大的模型，也不一定提供真值"}],"previousBlock":"b-2969278a28f21414a92667dc","nextBlock":"b-c00d9c4af0d4437b1974e41a","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c00d9c4af0d4437b1974e41a","type":"h2","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-c00d9c4af0d4437b1974e41a","content":"## 教师不一定是更大的模型，也不一定提供真值","sha256":"9daa40692cb1e71c17ff59655c00da11e6e36068fd5ca5549da5016ecb2f031e","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-bc5500b59c04052396feb1e7","nextBlock":"b-50253e0da6b6945bddea1b79","prerequisites":[{"id":"b-df469e5506abdeada1e506cd","url":"#b-df469e5506abdeada1e506cd","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-50253e0da6b6945bddea1b79","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-50253e0da6b6945bddea1b79","content":"传统知识蒸馏用训练好的教师给学生提供输出分布。教师 logits $a=(4,2,0)$ 对应三个类别，温度 $T$ 下的软目标为 $q_i=\\exp(a_i/T)/\\sum_j\\exp(a_j/T)$。$T=1$ 得到约 $(0.8668,0.1173,0.0159)$，$T=2$ 得到 $(0.6652,0.2447,0.0900)$。硬标签只指出第一类；软目标还表达第二类比第三类更接近教师判断。学生可用交叉熵 $-\\sum_iq_i\\log p_i$ 或等价的、忽略教师熵常数后的 KL 项拟合。","sha256":"60c0021cf3e308ab09b33e2eaa8dcc837399ca1d7ea911e4c5988bde77692bd7","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"a=(4,2,0)","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"T","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"q_i=\\exp(a_i/T)/\\sum_j\\exp(a_j/T)","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"T=1","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"(0.8668,0.1173,0.0159)","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"T=2","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"(0.6652,0.2447,0.0900)","definitionContext":"教师不一定是更大的模型，也不一定提供真值"},{"latex":"-\\sum_iq_i\\log p_i","definitionContext":"教师不一定是更大的模型，也不一定提供真值"}],"previousBlock":"b-c00d9c4af0d4437b1974e41a","nextBlock":"b-09745ff18aff64962f40883e","prerequisites":[{"id":"b-c00d9c4af0d4437b1974e41a","url":"#b-c00d9c4af0d4437b1974e41a","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-09745ff18aff64962f40883e","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-09745ff18aff64962f40883e","content":"蒸馏常把软目标项与真实标签项结合，并对温度目标作 $T^2$ 缩放以补偿典型梯度尺度变化。温度不是提高事实正确率的按钮；教师若依赖背景，软分布同样可能传递背景捷径。这里传递的是教师行为中的结构，是否有益需要在目标分布上测量。见 [Hinton 等，2015：蒸馏与温度软目标](https://arxiv.org/abs/1503.02531)。","sha256":"75a4add903cc4a1e6fb297730fba5165423f8e20dbba8243cadb62f713b8be20","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"T^2","definitionContext":"教师不一定是更大的模型，也不一定提供真值"}],"previousBlock":"b-50253e0da6b6945bddea1b79","nextBlock":"b-9fcc8e72ee0f8dd25e4c584a","prerequisites":[{"id":"b-c00d9c4af0d4437b1974e41a","url":"#b-c00d9c4af0d4437b1974e41a","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-9fcc8e72ee0f8dd25e4c584a","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-9fcc8e72ee0f8dd25e4c584a","content":"自监督教师则可以来自学生自身的历史参数。以 EMA 教师为例，教师参数 $\\xi\\leftarrow m\\xi+(1-m)\\theta$，目标分支停止梯度，学生去预测另一视图的教师输出。这节省人工标签，却没有凭空增加关于类别的权威来源。固定预训练教师、监督蒸馏教师和 EMA 自监督教师的来源与更新方式不同；看到“teacher”一词，先问它由什么数据训练、是否更新、哪条分支接收梯度。","sha256":"792d216843f4f01968151809602fa8c84963defa7cf462797140948c3f044a6b","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"\\xi\\leftarrow m\\xi+(1-m)\\theta","definitionContext":"教师不一定是更大的模型，也不一定提供真值"}],"previousBlock":"b-09745ff18aff64962f40883e","nextBlock":"b-7ff5ba3b13efb7116ddb12f4","prerequisites":[{"id":"b-c00d9c4af0d4437b1974e41a","url":"#b-c00d9c4af0d4437b1974e41a","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7ff5ba3b13efb7116ddb12f4","type":"table","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-7ff5ba3b13efb7116ddb12f4","content":"| 设计 | 谁产生目标 | 目标是否随学生训练变化 | 一项必须核对的风险 |\n| --- | --- | --- | --- |\n| 固定教师蒸馏 | 已训练模型 | 通常不变 | 教师错误和偏差会被模仿 |\n| EMA 自监督教师 | 学生参数的移动平均 | 平滑变化 | 常量解或退化解是否受到约束 |\n| 人工类别监督 | 标注规则与标注者 | 数据集内通常固定 | 类别集合遗漏与标注噪声 |","sha256":"5bea10eb69b97fe9ff1f3306fc82766b9958278943003d0ce821c7862c98df24","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["设计","谁产生目标","目标是否随学生训练变化","一项必须核对的风险"],"rows":[["设计","谁产生目标","目标是否随学生训练变化","一项必须核对的风险"],["固定教师蒸馏","已训练模型","通常不变","教师错误和偏差会被模仿"],["EMA 自监督教师","学生参数的移动平均","平滑变化","常量解或退化解是否受到约束"],["人工类别监督","标注规则与标注者","数据集内通常固定","类别集合遗漏与标注噪声"]]},"previousBlock":"b-9fcc8e72ee0f8dd25e4c584a","nextBlock":"b-eba4d4299b15149ffb382cf1","prerequisites":[{"id":"b-c00d9c4af0d4437b1974e41a","url":"#b-c00d9c4af0d4437b1974e41a","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-eba4d4299b15149ffb382cf1","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/1861e17c9d7c49f0970e890873be4ff6.html#b-eba4d4299b15149ffb382cf1","content":"BYOL 使用在线/目标网络、预测器和停止梯度等不对称设计；DINO 使用教师—学生分布匹配，并结合 centering、sharpening 等机制。它们给出了不显式使用负例也能学表示的实证路线。[BYOL](https://arxiv.org/abs/2006.07733) · [DINO](https://arxiv.org/abs/2104.14294)。不能把结果压缩成“EMA 单独保证不坍塌”：若随意删掉预测器、改变归一化或温度，系统已不是论文中的训练组合；只观察训练损失也识别不了所有退化。","sha256":"ad1c9cdc8c931315cefa1f74029155573a31c600a5d190e736b1616f4743d31a","chapter":{"id":"course","title":"教师不一定是更大的模型，也不一定提供真值"},"unit":{"id":"notebook-blocks/1861e17c9d7c49f0970e890873be4ff6","title":"教师不一定是更大的模型，也不一定提供真值"},"page":{"id":"reading-opening","title":"教师不一定是更大的模型，也不一定提供真值","number":1},"sectionTitle":"教师不一定是更大的模型，也不一定提供真值","currentQuestion":"教师不一定是更大的模型，也不一定提供真值","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7ff5ba3b13efb7116ddb12f4","nextBlock":null,"prerequisites":[{"id":"b-c00d9c4af0d4437b1974e41a","url":"#b-c00d9c4af0d4437b1974e41a","role":"section-title"},{"id":"b-16cdf10e3925ad469a43f0d3","url":"#b-16cdf10e3925ad469a43f0d3","role":"problem-context"}],"relatedBlocks":[]}]}