{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"CLIP 选择的任务：先学哪幅图与哪段话相配","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html","markdown":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.md","pages":[{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","unit":"已知图文配对怎样变成可用的文字候选接口？","unitId":"clip-pairing-to-candidates","chapter":"III","number":1},{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","unit":"规模化图文监督带来了哪些可验证的进展？","unitId":"clip-data-and-transfer","chapter":"III","number":2}],"aliases":[{"oldId":"explanation","targetId":"clip-paired-training-answer","operation":"split-explanation-range"},{"oldId":"b-1ba8cc89a291f0c91ec42a23","targetId":"b-1ba8cc89a291f0c91ec42a23","operation":"historical-explanation","sourceScope":"clip-transfer-evidence"},{"oldId":"b-92cf5fee8e408ea0a3e89c96","targetId":"b-92cf5fee8e408ea0a3e89c96","operation":"historical-explanation","sourceScope":"document"},{"oldId":"b-2307e305d7d118173038b447","targetId":"b-2307e305d7d118173038b447","operation":"historical-explanation","sourceScope":"clip-paired-training-answer"},{"oldId":"b-0838cbf13e88d728234498ff","targetId":"b-0838cbf13e88d728234498ff","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","afterSHA256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","reason":"Same source body in authored semantic page"},{"oldId":"b-a4ef68a9ca667d4ecc37d1df","targetId":"b-a4ef68a9ca667d4ecc37d1df","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"4226b9b17aad23def00198e69f13746e1a1a09082472f11e643c88821fdf1e1a","afterSHA256":"4226b9b17aad23def00198e69f13746e1a1a09082472f11e643c88821fdf1e1a","reason":"Same source body in authored semantic page"},{"oldId":"b-ec069210cc0b7dc9f041b595","targetId":"b-ec069210cc0b7dc9f041b595","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"0d653233ba094cef5ff2f1272d447699b1f92f65dbfb71291aa002592f5e7e70","afterSHA256":"0d653233ba094cef5ff2f1272d447699b1f92f65dbfb71291aa002592f5e7e70","reason":"Same source body in authored semantic page"},{"oldId":"b-e6eefa3edbba4a788e613f0b","targetId":"b-e6eefa3edbba4a788e613f0b","page":"clip-paired-training-answer","operation":"rewrite","beforeSHA256":"3a44fc05be8824925f0b25fde514b6f95a9166e4aa6b9c6e7a2417b7f2477f16","afterSHA256":"e978a991267249d3f485cc06cae5a17430320dea5103778b7ebefdf9b3904a47","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-75adba520c71f440f4fd5f03","targetId":"b-75adba520c71f440f4fd5f03","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","afterSHA256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","reason":"Same source body in authored semantic page"},{"oldId":"b-1f04e9aad042b12b11da969c","targetId":"b-1f04e9aad042b12b11da969c","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"81bb4df55fead984006a16d89cc930df0d06075a953c7c70297a5a5c2d361ad3","afterSHA256":"81bb4df55fead984006a16d89cc930df0d06075a953c7c70297a5a5c2d361ad3","reason":"Same source body in authored semantic page"},{"oldId":"b-20f1c24230c34386b08a60c1","targetId":"b-20f1c24230c34386b08a60c1","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"8fb7ae85ce84fc4aaa608a88964f1f5a3b8455936cd99fc5e88dff8169678c5d","afterSHA256":"8fb7ae85ce84fc4aaa608a88964f1f5a3b8455936cd99fc5e88dff8169678c5d","reason":"Same source body in authored semantic page"},{"oldId":"b-b0ed2f34449d31f36ea80c27","targetId":"b-b0ed2f34449d31f36ea80c27","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"b9680e25d4e832aad1b22242c17efb2bca353c25fa2991448a62994107fab30e","afterSHA256":"b9680e25d4e832aad1b22242c17efb2bca353c25fa2991448a62994107fab30e","reason":"Same source body in authored semantic page"},{"oldId":"b-62ccd839c991d7c548ff66b0","targetId":"b-62ccd839c991d7c548ff66b0","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"740e3b6ade773aea1d7a912a98f2a7c09f126fc75b8b574ca9d886cfe6c40342","afterSHA256":"740e3b6ade773aea1d7a912a98f2a7c09f126fc75b8b574ca9d886cfe6c40342","reason":"Same source body in authored semantic page"},{"oldId":"b-6586029c6397c3c3c813b2ac","targetId":"b-6586029c6397c3c3c813b2ac","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"bf1d8692ca1aa8689a2052607dbacad6413518129e70c31345ccf4c368ef2502","afterSHA256":"bf1d8692ca1aa8689a2052607dbacad6413518129e70c31345ccf4c368ef2502","reason":"Same source body in authored semantic page"},{"oldId":"b-03f18e1814660d49f25c3678","targetId":"b-03f18e1814660d49f25c3678","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"3efe21234482d01574f0859dc902c0d8f0ab78268142b8dfaf90b6d5dfca8633","afterSHA256":"3efe21234482d01574f0859dc902c0d8f0ab78268142b8dfaf90b6d5dfca8633","reason":"Same source body in authored semantic page"},{"oldId":"b-9d69fe51cc46bc582de0d7b4","targetId":"b-9d69fe51cc46bc582de0d7b4","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"64e3583e1614e56e2bd4c2d343c1dbd40fbbe74d1049595e20094ef36ead804c","afterSHA256":"64e3583e1614e56e2bd4c2d343c1dbd40fbbe74d1049595e20094ef36ead804c","reason":"Same source body in authored semantic page"},{"oldId":"b-791e0a9eadbe9fa7bbf26859","targetId":"b-791e0a9eadbe9fa7bbf26859","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"2145d99c9fb593c4d0d8ee18721dfc9184b3fbedd0835c31084a6d2b740a47a4","afterSHA256":"2145d99c9fb593c4d0d8ee18721dfc9184b3fbedd0835c31084a6d2b740a47a4","reason":"Same source body in authored semantic page"},{"oldId":"b-e0a3645e3282e9292d363fa1","targetId":"b-e0a3645e3282e9292d363fa1","page":"clip-paired-training-answer","operation":"move","beforeSHA256":"2145d99c9fb593c4d0d8ee18721dfc9184b3fbedd0835c31084a6d2b740a47a4","afterSHA256":"2145d99c9fb593c4d0d8ee18721dfc9184b3fbedd0835c31084a6d2b740a47a4","reason":"Same source body in authored semantic page"},{"oldId":"b-ffd2c16a950756775ec3af85","targetId":"b-ffd2c16a950756775ec3af85","page":"clip-transfer-evidence","operation":"move","beforeSHA256":"32fba3360fa674f66fececa5b66b13a3802216b91567c9dd5c1681cce7f2c586","afterSHA256":"32fba3360fa674f66fececa5b66b13a3802216b91567c9dd5c1681cce7f2c586","reason":"Same source body in authored semantic page"},{"oldId":"b-2d68eeafa7d82d8d92ca8178","targetId":"b-2d68eeafa7d82d8d92ca8178","page":"clip-transfer-evidence","operation":"move","beforeSHA256":"6d7f0dd74b4667c6a08b0646dce855f483d825be4b813261f23dc52e0afb3b1f","afterSHA256":"6d7f0dd74b4667c6a08b0646dce855f483d825be4b813261f23dc52e0afb3b1f","reason":"Same source body in authored semantic page"},{"oldId":"b-a8f4025bbfd1289089de103c","targetId":"b-a8f4025bbfd1289089de103c","page":"clip-transfer-evidence","operation":"move","beforeSHA256":"1212618a46874991135f8b45069f106d46e0cb3f6ac860805804ed731db72c91","afterSHA256":"1212618a46874991135f8b45069f106d46e0cb3f6ac860805804ed731db72c91","reason":"Same source body in authored semantic page"},{"oldId":"b-7c6fad429df724ae709e8e11","targetId":"b-7c6fad429df724ae709e8e11","page":"clip-transfer-evidence","operation":"move","beforeSHA256":"1ac2ccaaed2600cd171432e84791b757c5c816d12897751f508cab63ff1af1dc","afterSHA256":"1ac2ccaaed2600cd171432e84791b757c5c816d12897751f508cab63ff1af1dc","reason":"Same source body in authored semantic page"},{"oldId":"b-888e014b2e6b70424625714f","targetId":"b-888e014b2e6b70424625714f","page":"clip-transfer-evidence","operation":"move","beforeSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","afterSHA256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","reason":"Same source body in authored semantic page"},{"oldId":"b-34891a32c50e15172affdbac","targetId":"b-34891a32c50e15172affdbac","page":"clip-transfer-evidence","operation":"move","beforeSHA256":"166916ad1183e6f9502f09680e3c46d03645f22b206ed9dd396bec1e4ae95fc1","afterSHA256":"166916ad1183e6f9502f09680e3c46d03645f22b206ed9dd396bec1e4ae95fc1","reason":"Same source body in authored semantic page"},{"oldId":"clip-paired-training-answer-title","targetId":"b-872a6492b99627ef880e8ac9","operation":"retain-source-identity"},{"oldId":"clip-transfer-evidence-title","targetId":"b-0a2e573cd06ce203ad147fb9","operation":"retain-source-identity"},{"oldId":"clip-transfer-evidence-bridge","targetId":"b-98ef756baa3f71cb3069d417","operation":"retain-source-identity"}],"blocks":[{"id":"b-872a6492b99627ef880e8ac9","type":"h2","scope":"clip-paired-training-answer","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-872a6492b99627ef880e8ac9","content":"## 从固定标签表到已知图文配对","sha256":"9c9509e85ae587d58def021a5e0d64480f8a2f0f1d4ff181c6caa8745b5d2a00","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-0838cbf13e88d728234498ff","prerequisites":[{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-0838cbf13e88d728234498ff","type":"h2","scope":"explanation","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-0838cbf13e88d728234498ff","content":"## 解释、算例与证据","sha256":"d71cf15e707c8a3674b4c90a599efee783cb84da5b8c8236a1d01c3add7b4958","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-872a6492b99627ef880e8ac9","nextBlock":"b-a4ef68a9ca667d4ecc37d1df","prerequisites":[{"id":"b-872a6492b99627ef880e8ac9","url":"#b-872a6492b99627ef880e8ac9","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-a4ef68a9ca667d4ecc37d1df","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-a4ef68a9ca667d4ecc37d1df","content":"# P56 · CLIP 选择的任务：先学哪幅图与哪段话相配","sha256":"4226b9b17aad23def00198e69f13746e1a1a09082472f11e643c88821fdf1e1a","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0838cbf13e88d728234498ff","nextBlock":"b-ec069210cc0b7dc9f041b595","prerequisites":[{"id":"b-0838cbf13e88d728234498ff","url":"#b-0838cbf13e88d728234498ff","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-ec069210cc0b7dc9f041b595","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-ec069210cc0b7dc9f041b595","content":"CLIP 选择配对识别作为学习任务。两套编码器各自处理图片和文字，配对资料指出哪些对应应更容易被选出。与图像自监督对比的联系是训练关系的来源不同；文字监督拓展语义供给，下一页才用矩阵落实损失。","sha256":"0d653233ba094cef5ff2f1272d447699b1f92f65dbfb71291aa002592f5e7e70","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a4ef68a9ca667d4ecc37d1df","nextBlock":"b-e6eefa3edbba4a788e613f0b","prerequisites":[{"id":"b-a4ef68a9ca667d4ecc37d1df","url":"#b-a4ef68a9ca667d4ecc37d1df","role":"section-title"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-e6eefa3edbba4a788e613f0b","type":"figure","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-e6eefa3edbba4a788e613f0b","content":"图内标签与数值：图像编码器；“红衣人拿着篮球，；旁边站着蓝衣人。”；文字编码器；文字 → 向量；比较；训练答案：这一图像与这一段文字配成一对；向量条纹仅标示接口，未作定量含义；CLIP 学习图文匹配，不在此分支生成完整句子。\n\n图像 Iᵢ → 图像编码器 f → 单位向量 vᵢ；配对文字 Tᵢ → 文字编码器 g → 单位向量 uᵢ。\n\n比较 Sᵢⱼ = vᵢᵀuⱼ。已知图文配对提供训练答案；无需先生成句子。图中条纹表示接口，不是实测分数。\n图像和文字分别编码成向量，在共享空间比较；图文配对提供训练答案。 [查看对应图解状态](/#/l3-r2-p56/0)[查看完整图解](/course/figures/l3-r2-p56-0.html)","sha256":"e978a991267249d3f485cc06cae5a17430320dea5103778b7ebefdf9b3904a47","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"figure":{"caption":"图像和文字分别编码成向量，在共享空间比较；图文配对提供训练答案。 查看对应图解状态","labels":["图像编码器","“红衣人拿着篮球，","旁边站着蓝衣人。”","文字编码器","文字 → 向量","比较","训练答案：这一图像与这一段文字配成一对","向量条纹仅标示接口，未作定量含义；CLIP 学习图文匹配，不在此分支生成完整句子。"],"description":"图像 Iᵢ → 图像编码器 f → 单位向量 vᵢ；配对文字 Tᵢ → 文字编码器 g → 单位向量 uᵢ。比较 Sᵢⱼ = vᵢᵀuⱼ。已知图文配对提供训练答案；无需先生成句子。图中条纹表示接口，不是实测分数。","evidenceIdentity":"图像和文字分别编码成向量，在共享空间比较；图文配对提供训练答案。 查看对应图解状态","images":[{"src":"/assets/display/512610a019c560ff0384e86a.webp","alt":""}]},"previousBlock":"b-ec069210cc0b7dc9f041b595","nextBlock":"b-75adba520c71f440f4fd5f03","prerequisites":[{"id":"b-a4ef68a9ca667d4ecc37d1df","url":"#b-a4ef68a9ca667d4ecc37d1df","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-75adba520c71f440f4fd5f03","type":"h2","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-75adba520c71f440f4fd5f03","content":"## 课堂讲述","sha256":"786ce10a6b32a492259a3dccd0a54e61cd41d28a34382fb7b5386c5f03b7ad69","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-e6eefa3edbba4a788e613f0b","nextBlock":"b-1f04e9aad042b12b11da969c","prerequisites":[{"id":"b-a4ef68a9ca667d4ecc37d1df","url":"#b-a4ef68a9ca667d4ecc37d1df","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-1f04e9aad042b12b11da969c","type":"p","scope":"narrative","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-1f04e9aad042b12b11da969c","content":"如果希望利用更多图文配对，不一定非要从图片预测描述中的每一个准确词语。CLIP 选择了更直接的对应任务：让配对图文的表示相互匹配。\n这里我们可以复用第一部分的理解。那里比较的是同一图像产生的不同视图；这里比较的是图像与文字。训练目标仍然围绕对应关系，但文字带来了人已经组织过的语义。\n两边各有编码器，并不是先把图片翻译成完整句子，再拿句子比较。编码器把它们送进可比较的表示空间。这个任务到底怎样形成监督，下一页用一个很小的批次完整走一遍。","sha256":"81bb4df55fead984006a16d89cc930df0d06075a953c7c70297a5a5c2d361ad3","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-75adba520c71f440f4fd5f03","nextBlock":"b-20f1c24230c34386b08a60c1","prerequisites":[{"id":"b-75adba520c71f440f4fd5f03","url":"#b-75adba520c71f440f4fd5f03","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-20f1c24230c34386b08a60c1","type":"h3","scope":"clip-pairing-objective","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-20f1c24230c34386b08a60c1","content":"### CLIP：文字怎样从标签表变成学习信号","sha256":"8fb7ae85ce84fc4aaa608a88964f1f5a3b8455936cd99fc5e88dff8169678c5d","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"CLIP：文字怎样从标签表变成学习信号","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":"clip-pairing-objective","formulas":[],"previousBlock":"b-1f04e9aad042b12b11da969c","nextBlock":"b-b0ed2f34449d31f36ea80c27","prerequisites":[{"id":"b-75adba520c71f440f4fd5f03","url":"#b-75adba520c71f440f4fd5f03","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"},{"id":"b-b0ed2f34449d31f36ea80c27","url":"#b-b0ed2f34449d31f36ea80c27","relation":"same-reasoning"},{"id":"b-62ccd839c991d7c548ff66b0","url":"#b-62ccd839c991d7c548ff66b0","relation":"same-reasoning"},{"id":"b-6586029c6397c3c3c813b2ac","url":"#b-6586029c6397c3c3c813b2ac","relation":"same-reasoning"},{"id":"b-03f18e1814660d49f25c3678","url":"#b-03f18e1814660d49f25c3678","relation":"same-reasoning"}]},{"id":"b-b0ed2f34449d31f36ea80c27","type":"p","scope":"clip-pairing-objective","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-b0ed2f34449d31f36ea80c27","content":"监督分类先规定一个类别表，再让模型预测表中的索引。图文配对可以提供更丰富的语义：文字可能同时包含对象、属性、动作和场景。CLIP 选择一个可扩展的任务——在一批图文中识别原有配对——把这些描述用于训练可比较的表示。","sha256":"b9680e25d4e832aad1b22242c17efb2bca353c25fa2991448a62994107fab30e","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"CLIP：文字怎样从标签表变成学习信号","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":"clip-pairing-objective","formulas":[],"previousBlock":"b-20f1c24230c34386b08a60c1","nextBlock":"b-62ccd839c991d7c548ff66b0","prerequisites":[{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"},{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","relation":"same-reasoning"},{"id":"b-62ccd839c991d7c548ff66b0","url":"#b-62ccd839c991d7c548ff66b0","relation":"same-reasoning"},{"id":"b-6586029c6397c3c3c813b2ac","url":"#b-6586029c6397c3c3c813b2ac","relation":"same-reasoning"},{"id":"b-03f18e1814660d49f25c3678","url":"#b-03f18e1814660d49f25c3678","relation":"same-reasoning"}]},{"id":"b-62ccd839c991d7c548ff66b0","type":"p","scope":"clip-pairing-objective","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-62ccd839c991d7c548ff66b0","content":"设图像编码器为 $f$，文字编码器为 $g$，归一化后得到 $v_i=f(I_i)/\\|f(I_i)\\|$、$u_j=g(T_j)/\\|g(T_j)\\|$。两个编码器分别处理各自输入，然后用点积 $S_{ij}=v_i^Tu_j$ 比较。没有先让图像生成一句话，再把两句话相互比较。","sha256":"740e3b6ade773aea1d7a912a98f2a7c09f126fc75b8b574ca9d886cfe6c40342","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"CLIP：文字怎样从标签表变成学习信号","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":"clip-pairing-objective","formulas":[{"latex":"f","definitionContext":"监督分类先规定一个类别表，再让模型预测表中的索引。图文配对可以提供更丰富的语义：文字可能同时包含对象、属性、动作和场景。CLIP 选择一个可扩展的任务——在一批图文中识别原有配对——把这些描述用于训练可比较的表示。\n设图像编码器为 $f$，文字编码器为 $g$，归一化后得到 $v_i=f(I_i)/\\|f(I_i)\\|$、$u_j=g(T_j)/\\|g(T_j)\\|$。两个编码器分别处理各自输入，然后用点积 $S_{ij}=v_i^Tu_j$ 比较。没有先让图像生成一句话，再把两句话相互比较。\n与第一部分的图像对比学习相连：两张增强视图的正关系来自同一原图；CLIP 的正关系来自资料中的图文配对。文字已经包含人的选择和表达，因此“没有固定类别标签”不等于“没有人类监督”。\n训练完保留两套编码器，新图像与新文字可以进入同一比较接口。使用者能更换候选描述或建立检索库，这是方法的正面能力。它还没有提供下一词预测分布或框回归输出；若新任务需要这些输出，要另行定义模块、目标和训练。"},{"latex":"g","definitionContext":"监督分类先规定一个类别表，再让模型预测表中的索引。图文配对可以提供更丰富的语义：文字可能同时包含对象、属性、动作和场景。CLIP 选择一个可扩展的任务——在一批图文中识别原有配对——把这些描述用于训练可比较的表示。\n设图像编码器为 $f$，文字编码器为 $g$，归一化后得到 $v_i=f(I_i)/\\|f(I_i)\\|$、$u_j=g(T_j)/\\|g(T_j)\\|$。两个编码器分别处理各自输入，然后用点积 $S_{ij}=v_i^Tu_j$ 比较。没有先让图像生成一句话，再把两句话相互比较。\n与第一部分的图像对比学习相连：两张增强视图的正关系来自同一原图；CLIP 的正关系来自资料中的图文配对。文字已经包含人的选择和表达，因此“没有固定类别标签”不等于“没有人类监督”。\n训练完保留两套编码器，新图像与新文字可以进入同一比较接口。使用者能更换候选描述或建立检索库，这是方法的正面能力。它还没有提供下一词预测分布或框回归输出；若新任务需要这些输出，要另行定义模块、目标和训练。"},{"latex":"v_i=f(I_i)/\\|f(I_i)\\|","definitionContext":"监督分类先规定一个类别表，再让模型预测表中的索引。图文配对可以提供更丰富的语义：文字可能同时包含对象、属性、动作和场景。CLIP 选择一个可扩展的任务——在一批图文中识别原有配对——把这些描述用于训练可比较的表示。\n设图像编码器为 $f$，文字编码器为 $g$，归一化后得到 $v_i=f(I_i)/\\|f(I_i)\\|$、$u_j=g(T_j)/\\|g(T_j)\\|$。两个编码器分别处理各自输入，然后用点积 $S_{ij}=v_i^Tu_j$ 比较。没有先让图像生成一句话，再把两句话相互比较。\n与第一部分的图像对比学习相连：两张增强视图的正关系来自同一原图；CLIP 的正关系来自资料中的图文配对。文字已经包含人的选择和表达，因此“没有固定类别标签”不等于“没有人类监督”。\n训练完保留两套编码器，新图像与新文字可以进入同一比较接口。使用者能更换候选描述或建立检索库，这是方法的正面能力。它还没有提供下一词预测分布或框回归输出；若新任务需要这些输出，要另行定义模块、目标和训练。"},{"latex":"u_j=g(T_j)/\\|g(T_j)\\|","definitionContext":"监督分类先规定一个类别表，再让模型预测表中的索引。图文配对可以提供更丰富的语义：文字可能同时包含对象、属性、动作和场景。CLIP 选择一个可扩展的任务——在一批图文中识别原有配对——把这些描述用于训练可比较的表示。\n设图像编码器为 $f$，文字编码器为 $g$，归一化后得到 $v_i=f(I_i)/\\|f(I_i)\\|$、$u_j=g(T_j)/\\|g(T_j)\\|$。两个编码器分别处理各自输入，然后用点积 $S_{ij}=v_i^Tu_j$ 比较。没有先让图像生成一句话，再把两句话相互比较。\n与第一部分的图像对比学习相连：两张增强视图的正关系来自同一原图；CLIP 的正关系来自资料中的图文配对。文字已经包含人的选择和表达，因此“没有固定类别标签”不等于“没有人类监督”。\n训练完保留两套编码器，新图像与新文字可以进入同一比较接口。使用者能更换候选描述或建立检索库，这是方法的正面能力。它还没有提供下一词预测分布或框回归输出；若新任务需要这些输出，要另行定义模块、目标和训练。"},{"latex":"S_{ij}=v_i^Tu_j","definitionContext":"监督分类先规定一个类别表，再让模型预测表中的索引。图文配对可以提供更丰富的语义：文字可能同时包含对象、属性、动作和场景。CLIP 选择一个可扩展的任务——在一批图文中识别原有配对——把这些描述用于训练可比较的表示。\n设图像编码器为 $f$，文字编码器为 $g$，归一化后得到 $v_i=f(I_i)/\\|f(I_i)\\|$、$u_j=g(T_j)/\\|g(T_j)\\|$。两个编码器分别处理各自输入，然后用点积 $S_{ij}=v_i^Tu_j$ 比较。没有先让图像生成一句话，再把两句话相互比较。\n与第一部分的图像对比学习相连：两张增强视图的正关系来自同一原图；CLIP 的正关系来自资料中的图文配对。文字已经包含人的选择和表达，因此“没有固定类别标签”不等于“没有人类监督”。\n训练完保留两套编码器，新图像与新文字可以进入同一比较接口。使用者能更换候选描述或建立检索库，这是方法的正面能力。它还没有提供下一词预测分布或框回归输出；若新任务需要这些输出，要另行定义模块、目标和训练。"}],"previousBlock":"b-b0ed2f34449d31f36ea80c27","nextBlock":"b-6586029c6397c3c3c813b2ac","prerequisites":[{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"},{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","relation":"same-reasoning"},{"id":"b-b0ed2f34449d31f36ea80c27","url":"#b-b0ed2f34449d31f36ea80c27","relation":"same-reasoning"},{"id":"b-6586029c6397c3c3c813b2ac","url":"#b-6586029c6397c3c3c813b2ac","relation":"same-reasoning"},{"id":"b-03f18e1814660d49f25c3678","url":"#b-03f18e1814660d49f25c3678","relation":"same-reasoning"}]},{"id":"b-6586029c6397c3c3c813b2ac","type":"p","scope":"clip-pairing-objective","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-6586029c6397c3c3c813b2ac","content":"与第一部分的图像对比学习相连：两张增强视图的正关系来自同一原图；CLIP 的正关系来自资料中的图文配对。文字已经包含人的选择和表达，因此“没有固定类别标签”不等于“没有人类监督”。","sha256":"bf1d8692ca1aa8689a2052607dbacad6413518129e70c31345ccf4c368ef2502","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"CLIP：文字怎样从标签表变成学习信号","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":"clip-pairing-objective","formulas":[],"previousBlock":"b-62ccd839c991d7c548ff66b0","nextBlock":"b-03f18e1814660d49f25c3678","prerequisites":[{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"},{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","relation":"same-reasoning"},{"id":"b-b0ed2f34449d31f36ea80c27","url":"#b-b0ed2f34449d31f36ea80c27","relation":"same-reasoning"},{"id":"b-62ccd839c991d7c548ff66b0","url":"#b-62ccd839c991d7c548ff66b0","relation":"same-reasoning"},{"id":"b-03f18e1814660d49f25c3678","url":"#b-03f18e1814660d49f25c3678","relation":"same-reasoning"}]},{"id":"b-03f18e1814660d49f25c3678","type":"p","scope":"clip-pairing-objective","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-03f18e1814660d49f25c3678","content":"训练完保留两套编码器，新图像与新文字可以进入同一比较接口。使用者能更换候选描述或建立检索库，这是方法的正面能力。它还没有提供下一词预测分布或框回归输出；若新任务需要这些输出，要另行定义模块、目标和训练。","sha256":"3efe21234482d01574f0859dc902c0d8f0ab78268142b8dfaf90b6d5dfca8633","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"CLIP：文字怎样从标签表变成学习信号","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":"clip-pairing-objective","formulas":[],"previousBlock":"b-6586029c6397c3c3c813b2ac","nextBlock":"b-a6eda2a541b358222bc5f692","prerequisites":[{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"},{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","relation":"same-reasoning"},{"id":"b-b0ed2f34449d31f36ea80c27","url":"#b-b0ed2f34449d31f36ea80c27","relation":"same-reasoning"},{"id":"b-62ccd839c991d7c548ff66b0","url":"#b-62ccd839c991d7c548ff66b0","relation":"same-reasoning"},{"id":"b-6586029c6397c3c3c813b2ac","url":"#b-6586029c6397c3c3c813b2ac","relation":"same-reasoning"}]},{"id":"b-a6eda2a541b358222bc5f692","type":"h4","scope":"clip-paired-training-answer","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-a6eda2a541b358222bc5f692","content":"#### 图示条件与公式约定","sha256":"ef7f915c1ae63ed67bd23fecf050249273ba0efa8965d2c8d1c1c6b2d3178122","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-03f18e1814660d49f25c3678","nextBlock":"b-9d69fe51cc46bc582de0d7b4","prerequisites":[{"id":"b-20f1c24230c34386b08a60c1","url":"#b-20f1c24230c34386b08a60c1","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-9d69fe51cc46bc582de0d7b4","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-9d69fe51cc46bc582de0d7b4","content":"向量条纹只表示不同模态的向量接口，条纹高度没有模型分数含义。图文配对提供已知的训练对应。","sha256":"64e3583e1614e56e2bd4c2d343c1dbd40fbbe74d1049595e20094ef36ead804c","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-a6eda2a541b358222bc5f692","nextBlock":"b-ff275e2e4485f7f9ea04c469","prerequisites":[{"id":"b-a6eda2a541b358222bc5f692","url":"#b-a6eda2a541b358222bc5f692","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-ff275e2e4485f7f9ea04c469","type":"h4","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-ff275e2e4485f7f9ea04c469","content":"#### 这条线索的展开与后续阅读","sha256":"48b8229f7559b36ed4d5628416e3991e5c73293fbce43374db67386c8736f3bc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-9d69fe51cc46bc582de0d7b4","nextBlock":"b-791e0a9eadbe9fa7bbf26859","prerequisites":[{"id":"b-a6eda2a541b358222bc5f692","url":"#b-a6eda2a541b358222bc5f692","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-791e0a9eadbe9fa7bbf26859","type":"ul","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-791e0a9eadbe9fa7bbf26859","content":"- [CLIP 从图文对走到候选接口｜问题从哪里出现](/supplement.html?return=l3-r2-p56/0#/l3-clip-full-chain-problem/0) 本页说明","sha256":"2145d99c9fb593c4d0d8ee18721dfc9184b3fbedd0835c31084a6d2b740a47a4","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ff275e2e4485f7f9ea04c469","nextBlock":"b-e0a3645e3282e9292d363fa1","prerequisites":[{"id":"b-ff275e2e4485f7f9ea04c469","url":"#b-ff275e2e4485f7f9ea04c469","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-e0a3645e3282e9292d363fa1","type":"li","scope":"retained-reading","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-e0a3645e3282e9292d363fa1","content":"- [CLIP 从图文对走到候选接口｜问题从哪里出现](/supplement.html?return=l3-r2-p56/0#/l3-clip-full-chain-problem/0) 本页说明","sha256":"2145d99c9fb593c4d0d8ee18721dfc9184b3fbedd0835c31084a6d2b740a47a4","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-pairing-to-candidates","title":"已知图文配对怎样变成可用的文字候选接口？"},"page":{"id":"clip-paired-training-answer","title":"从固定标签表到已知图文配对","number":1},"sectionTitle":"从固定标签表到已知图文配对","currentQuestion":"从一对输入到双向损失，再到真实候选干预，哪些量由训练决定、哪些由使用者决定？","sourceSlides":["l3-r2-p56"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-791e0a9eadbe9fa7bbf26859","nextBlock":null,"prerequisites":[{"id":"b-ff275e2e4485f7f9ea04c469","url":"#b-ff275e2e4485f7f9ea04c469","role":"section-title"},{"id":"b-ec069210cc0b7dc9f041b595","url":"#b-ec069210cc0b7dc9f041b595","role":"problem-context"}],"relatedBlocks":[{"id":"b-e6eefa3edbba4a788e613f0b","url":"#b-e6eefa3edbba4a788e613f0b","relation":"figure-and-explanation"}]},{"id":"b-0a2e573cd06ce203ad147fb9","type":"h2","scope":"clip-transfer-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-0a2e573cd06ce203ad147fb9","content":"## 先确认语言指定任务的实质进展","sha256":"e4a5808ead3032876e5ec8b808f49796945f5e7a946069472d69c75bb186beec","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"先确认语言指定任务的实质进展","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-98ef756baa3f71cb3069d417","prerequisites":[{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-98ef756baa3f71cb3069d417","type":"p","scope":"clip-transfer-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-98ef756baa3f71cb3069d417","content":"规模提供机会，但进展还要由明确协议下的迁移结果支持。","sha256":"8017403ed296a6584280fd34991b8e1dfb05089367d532635c085af4b1fcff29","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"先确认语言指定任务的实质进展","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0a2e573cd06ce203ad147fb9","nextBlock":"b-ffd2c16a950756775ec3af85","prerequisites":[{"id":"b-0a2e573cd06ce203ad147fb9","url":"#b-0a2e573cd06ce203ad147fb9","role":"section-title"}],"relatedBlocks":[]},{"id":"b-ffd2c16a950756775ec3af85","type":"h3","scope":"clip-positive-transfer-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-ffd2c16a950756775ec3af85","content":"### 正面证据：自然语言监督确实拓宽了下游接口","sha256":"32fba3360fa674f66fececa5b66b13a3802216b91567c9dd5c1681cce7f2c586","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"正面证据：自然语言监督确实拓宽了下游接口","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":"clip-positive-transfer-evidence","formulas":[],"previousBlock":"b-98ef756baa3f71cb3069d417","nextBlock":"b-2d68eeafa7d82d8d92ca8178","prerequisites":[{"id":"b-0a2e573cd06ce203ad147fb9","url":"#b-0a2e573cd06ce203ad147fb9","role":"section-title"},{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[{"id":"b-2d68eeafa7d82d8d92ca8178","url":"#b-2d68eeafa7d82d8d92ca8178","relation":"same-reasoning"},{"id":"b-a8f4025bbfd1289089de103c","url":"#b-a8f4025bbfd1289089de103c","relation":"same-reasoning"},{"id":"b-7c6fad429df724ae709e8e11","url":"#b-7c6fad429df724ae709e8e11","relation":"same-reasoning"}]},{"id":"b-2d68eeafa7d82d8d92ca8178","type":"p","scope":"clip-positive-transfer-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-2d68eeafa7d82d8d92ca8178","content":"CLIP 的论文贡献不只是一种损失公式。原工作研究了 30 多个视觉数据集上的迁移，且报告其 zero-shot 模型在 ImageNet 上达到原始 ResNet-50 的分类准确率水平，而不使用该任务的 128 万张训练样本训练专用分类器。数字与比较属于原论文协议，不能从本课程猫图实验重新推出。","sha256":"6d7f0dd74b4667c6a08b0646dce855f483d825be4b813261f23dc52e0afb3b1f","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"正面证据：自然语言监督确实拓宽了下游接口","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":"clip-positive-transfer-evidence","formulas":[],"previousBlock":"b-ffd2c16a950756775ec3af85","nextBlock":"b-a8f4025bbfd1289089de103c","prerequisites":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","role":"section-title"},{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","relation":"same-reasoning"},{"id":"b-a8f4025bbfd1289089de103c","url":"#b-a8f4025bbfd1289089de103c","relation":"same-reasoning"},{"id":"b-7c6fad429df724ae709e8e11","url":"#b-7c6fad429df724ae709e8e11","relation":"same-reasoning"}]},{"id":"b-a8f4025bbfd1289089de103c","type":"p","scope":"clip-positive-transfer-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-a8f4025bbfd1289089de103c","content":"这项结果支持一个明确判断：大规模图文匹配学到的表示，可以通过文字描述定义一批下游分类任务，减少逐任务重新标注和训练固定分类头的需求。语言成为任务指定接口，是对“分类表只能在训练时固定”的实质拓展。","sha256":"1212618a46874991135f8b45069f106d46e0cb3f6ac860805804ed731db72c91","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"正面证据：自然语言监督确实拓宽了下游接口","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":"clip-positive-transfer-evidence","formulas":[],"previousBlock":"b-2d68eeafa7d82d8d92ca8178","nextBlock":"b-7c6fad429df724ae709e8e11","prerequisites":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","role":"section-title"},{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","relation":"same-reasoning"},{"id":"b-2d68eeafa7d82d8d92ca8178","url":"#b-2d68eeafa7d82d8d92ca8178","relation":"same-reasoning"},{"id":"b-7c6fad429df724ae709e8e11","url":"#b-7c6fad429df724ae709e8e11","relation":"same-reasoning"}]},{"id":"b-7c6fad429df724ae709e8e11","type":"p","scope":"clip-positive-transfer-evidence","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-7c6fad429df724ae709e8e11","content":"然后再问边界才有意义：候选描述写得好不好、预训练是否覆盖概念、目标数据分布是否变化，都会影响迁移。关系定位和自由生成还要求别的输出。不是因为 CLIP 什么也没有证明而要继续研究，而是它已经打开了可用接口，研究者因此可以追问更细的空间、关系和交互能力。","sha256":"1ac2ccaaed2600cd171432e84791b757c5c816d12897751f508cab63ff1af1dc","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"正面证据：自然语言监督确实拓宽了下游接口","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":"clip-positive-transfer-evidence","formulas":[],"previousBlock":"b-a8f4025bbfd1289089de103c","nextBlock":"b-888e014b2e6b70424625714f","prerequisites":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","role":"section-title"},{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","relation":"same-reasoning"},{"id":"b-2d68eeafa7d82d8d92ca8178","url":"#b-2d68eeafa7d82d8d92ca8178","relation":"same-reasoning"},{"id":"b-a8f4025bbfd1289089de103c","url":"#b-a8f4025bbfd1289089de103c","relation":"same-reasoning"}]},{"id":"b-888e014b2e6b70424625714f","type":"h2","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-888e014b2e6b70424625714f","content":"## 沿这个问题继续","sha256":"cf44fcc62782fb4f6bdc42c16c5187d57556a03cb7ce6065e7bd9c36d8e7d733","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"先确认语言指定任务的实质进展","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7c6fad429df724ae709e8e11","nextBlock":"b-34891a32c50e15172affdbac","prerequisites":[{"id":"b-ffd2c16a950756775ec3af85","url":"#b-ffd2c16a950756775ec3af85","role":"section-title"},{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-34891a32c50e15172affdbac","type":"p","scope":"practice","url":"https://codingai-lec03.pages.dev/course/pages/l3-r2-p56.html#b-34891a32c50e15172affdbac","content":"先回看本页画面并指出输入、条件和输出，再按“CLIP：文字怎样从标签表变成学习信号 → 正面证据：自然语言监督确实拓宽了下游接口”阅读。相关实验以其已保存结果为起点，需要动手时再顺序执行 Notebook；","sha256":"166916ad1183e6f9502f09680e3c46d03645f22b206ed9dd396bec1e4ae95fc1","chapter":{"id":"III","title":"视觉与语言"},"unit":{"id":"clip-data-and-transfer","title":"规模化图文监督带来了哪些可验证的进展？"},"page":{"id":"clip-transfer-evidence","title":"先确认语言指定任务的实质进展","number":2},"sectionTitle":"先确认语言指定任务的实质进展","currentQuestion":"模型、训练数据与使用接口分别贡献了什么？","sourceSlides":["l3-r2-p56","l3-r2-p60"],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-888e014b2e6b70424625714f","nextBlock":null,"prerequisites":[{"id":"b-888e014b2e6b70424625714f","url":"#b-888e014b2e6b70424625714f","role":"section-title"},{"id":"b-98ef756baa3f71cb3069d417","url":"#b-98ef756baa3f71cb3069d417","role":"problem-context"}],"relatedBlocks":[]}]}