{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"CLS 04 · 重建：约束、位置与表示评价","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html","markdown":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.md","pages":[{"id":"reading-opening","title":"CLS 04 · 重建：约束、位置与表示评价","unit":"CLS 04 · 重建：约束、位置与表示评价","unitId":"cls05-重建-词典pca自编码器与-mae","chapter":"course","number":1},{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","unit":"CLS 04 · 重建：约束、位置与表示评价","unitId":"cls05-重建-词典pca自编码器与-mae","chapter":"course","number":2},{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","unit":"CLS 04 · 重建：约束、位置与表示评价","unitId":"cls05-重建-词典pca自编码器与-mae","chapter":"course","number":3},{"id":"reading-block-de414705e5654ffa86fc46358756877d","title":"读到这里，可以如何继续？","unit":"CLS 04 · 重建：约束、位置与表示评价","unitId":"cls05-重建-词典pca自编码器与-mae","chapter":"course","number":4}],"aliases":[{"oldId":"b-35c476a7359c3dc3777adfe0","targetId":"b-35c476a7359c3dc3777adfe0","page":"reading-opening","operation":"move","beforeSHA256":"4b32e93b37622319bc268dfcb993d4399abc4ee7dffabf03be88fa420b57d5d0","afterSHA256":"4b32e93b37622319bc268dfcb993d4399abc4ee7dffabf03be88fa420b57d5d0","reason":"Same source body in authored semantic page"},{"oldId":"b-3beb81246265a9b6940ad0d4","targetId":"b-3beb81246265a9b6940ad0d4","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-122fa7c700d78847699dbeec","targetId":"b-122fa7c700d78847699dbeec","page":"reading-opening","operation":"move","beforeSHA256":"db36e128a2d5a5e8c2d5360cbf60671bab45cb5494bbdd81f27d6b7d95ea74e0","afterSHA256":"db36e128a2d5a5e8c2d5360cbf60671bab45cb5494bbdd81f27d6b7d95ea74e0","reason":"Same source body in authored semantic page"},{"oldId":"b-169d05e1e9f2b2d606e6f9e2","targetId":"b-169d05e1e9f2b2d606e6f9e2","page":"reading-opening","operation":"move","beforeSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","afterSHA256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","reason":"Same source body in authored semantic page"},{"oldId":"b-2affa291d1e3f6e8ab733866","targetId":"b-2affa291d1e3f6e8ab733866","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"8556351a0a818e5fa2ef8199862458dc0e7165abe7355c0f371bb6d655e593e9","afterSHA256":"8556351a0a818e5fa2ef8199862458dc0e7165abe7355c0f371bb6d655e593e9","reason":"Same source body in authored semantic page"},{"oldId":"b-b2c6794499589ae3caa596c6","targetId":"b-b2c6794499589ae3caa596c6","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"33a9d35da9bf638cbbb8b5e9354a73f8d26efaaa61832a0a1a404bed3367025a","afterSHA256":"33a9d35da9bf638cbbb8b5e9354a73f8d26efaaa61832a0a1a404bed3367025a","reason":"Same source body in authored semantic page"},{"oldId":"b-364cc5857021335d35b1b61d","targetId":"b-364cc5857021335d35b1b61d","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"2864bf0006511ecd508b2de87e5c323f9679695de8967deee935355762a68f4c","afterSHA256":"2864bf0006511ecd508b2de87e5c323f9679695de8967deee935355762a68f4c","reason":"Same source body in authored semantic page"},{"oldId":"b-c9ea0827b123142b7ef07795","targetId":"b-c9ea0827b123142b7ef07795","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"9e07fc295fe95877d90b697ad5fa48296f312a7ab8d0889774a5f486fd8d090d","afterSHA256":"9e07fc295fe95877d90b697ad5fa48296f312a7ab8d0889774a5f486fd8d090d","reason":"Same source body in authored semantic page"},{"oldId":"b-b7e8ad649ebc19896d40d215","targetId":"b-b7e8ad649ebc19896d40d215","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"a8be541fcaf19bfce2bf41f6779590be20eb214a5f4faf49be22dcf9a7d9bbc0","afterSHA256":"a8be541fcaf19bfce2bf41f6779590be20eb214a5f4faf49be22dcf9a7d9bbc0","reason":"Same source body in authored semantic page"},{"oldId":"b-1e21dee7d58adb4f25797677","targetId":"b-1e21dee7d58adb4f25797677","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"9a8088a16b7096a914d6ede16623634ce17a57528701ccdb6e9cb90e6d1c8e7f","afterSHA256":"9a8088a16b7096a914d6ede16623634ce17a57528701ccdb6e9cb90e6d1c8e7f","reason":"Same source body in authored semantic page"},{"oldId":"b-f2c2667742ed6f6fef0e5d65","targetId":"b-f2c2667742ed6f6fef0e5d65","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"ab7a10e9728afbecdcec79d3f12505658449566b8277e7632a8f41cdc23677f7","afterSHA256":"ab7a10e9728afbecdcec79d3f12505658449566b8277e7632a8f41cdc23677f7","reason":"Same source body in authored semantic page"},{"oldId":"b-578e14dc3bb335659b9dedfa","targetId":"b-578e14dc3bb335659b9dedfa","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"aef13a0897067ed8c716ad7ba8fabf84019f69a5ea2017415804af017d79c7b6","afterSHA256":"aef13a0897067ed8c716ad7ba8fabf84019f69a5ea2017415804af017d79c7b6","reason":"Same source body in authored semantic page"},{"oldId":"b-b661e33088f8d4d3a9e3e347","targetId":"b-b661e33088f8d4d3a9e3e347","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"b95322b58e86aab97b9239ea97f9fbcd76a279b78c510fe88e115594dd0cd9ea","afterSHA256":"b95322b58e86aab97b9239ea97f9fbcd76a279b78c510fe88e115594dd0cd9ea","reason":"Same source body in authored semantic page"},{"oldId":"b-e2a332133a492c1bf17355e3","targetId":"b-e2a332133a492c1bf17355e3","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"314d138cb51e798d2754f0d3a4ea9e9f5db2ea2051da32427b62b50db0088186","afterSHA256":"314d138cb51e798d2754f0d3a4ea9e9f5db2ea2051da32427b62b50db0088186","reason":"Same source body in authored semantic page"},{"oldId":"b-ee6425c0f1570c24b5dc8797","targetId":"b-ee6425c0f1570c24b5dc8797","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"da8a5c96a3492ad378af6c0cbf696fd54ccd53c2c29bbc8e794d30c3a07e225e","afterSHA256":"da8a5c96a3492ad378af6c0cbf696fd54ccd53c2c29bbc8e794d30c3a07e225e","reason":"Same source body in authored semantic page"},{"oldId":"b-34bb8503809a97fdc5af70e9","targetId":"b-34bb8503809a97fdc5af70e9","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"5e93b74f70085ca5ec3b76b71d57b2a105f5a18720df2082b57f0ac8f1df1499","afterSHA256":"5e93b74f70085ca5ec3b76b71d57b2a105f5a18720df2082b57f0ac8f1df1499","reason":"Same source body in authored semantic page"},{"oldId":"b-684445c71410a0f198f8b4fc","targetId":"b-684445c71410a0f198f8b4fc","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"7ed392f7e5f8093e27fad86ce7c3c1dd94875138594c09c1db623b85a4946a51","afterSHA256":"7ed392f7e5f8093e27fad86ce7c3c1dd94875138594c09c1db623b85a4946a51","reason":"Same source body in authored semantic page"},{"oldId":"b-73a8bff2d9f823e3ea9b8437","targetId":"b-73a8bff2d9f823e3ea9b8437","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"859cfeee5fb3ba73e84cd3892eaf1ea08e08ffef5ebf530789c7027945093b85","afterSHA256":"859cfeee5fb3ba73e84cd3892eaf1ea08e08ffef5ebf530789c7027945093b85","reason":"Same source body in authored semantic page"},{"oldId":"b-8ebb2b48c62b2d2cc69790d1","targetId":"b-8ebb2b48c62b2d2cc69790d1","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"1ec743964fe5a8b6a41542c2f230348bcf69b21f317b8a22ea2f1483ada27a4b","afterSHA256":"1ec743964fe5a8b6a41542c2f230348bcf69b21f317b8a22ea2f1483ada27a4b","reason":"Same source body in authored semantic page"},{"oldId":"b-64a3661efdaa684b3fc9aaf2","targetId":"b-64a3661efdaa684b3fc9aaf2","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"e62fcdbdf8f5d621235c6d665a17152c1736f9eb0ff056daa0c2d302d56ee703","afterSHA256":"e62fcdbdf8f5d621235c6d665a17152c1736f9eb0ff056daa0c2d302d56ee703","reason":"Same source body in authored semantic page"},{"oldId":"b-3c6fcb459cbbbf5fa64a3c4e","targetId":"b-3c6fcb459cbbbf5fa64a3c4e","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"4d82f52f7518bc4d90a45a49b8188143486de499bd9a390f4e9be7d7b2985f15","afterSHA256":"4d82f52f7518bc4d90a45a49b8188143486de499bd9a390f4e9be7d7b2985f15","reason":"Same source body in authored semantic page"},{"oldId":"b-d64fd917b95d2f2d11b93a45","targetId":"b-d64fd917b95d2f2d11b93a45","page":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","operation":"move","beforeSHA256":"54a1680c2cde89592947009c399bb9696954ae9ee53a7738e72503f17248e65a","afterSHA256":"54a1680c2cde89592947009c399bb9696954ae9ee53a7738e72503f17248e65a","reason":"Same source body in authored semantic page"},{"oldId":"b-70d6b80cfe4ac770a53f0b9a","targetId":"b-70d6b80cfe4ac770a53f0b9a","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"94c92248638a95e30f63c78bc247c7bac3e334d71bda920fd01554a003dbda6d","afterSHA256":"94c92248638a95e30f63c78bc247c7bac3e334d71bda920fd01554a003dbda6d","reason":"Same source body in authored semantic page"},{"oldId":"b-cb1376ea688d9d381bb3f36c","targetId":"b-cb1376ea688d9d381bb3f36c","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"11689a6e9d1b70e1f58e6c9dc369e5b3e87fa0cb70c4c6f28d85cbea174eb60a","afterSHA256":"11689a6e9d1b70e1f58e6c9dc369e5b3e87fa0cb70c4c6f28d85cbea174eb60a","reason":"Same source body in authored semantic page"},{"oldId":"b-61d52de149887e22ba65ab3e","targetId":"b-61d52de149887e22ba65ab3e","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"5ac484a3f720ac4cf71140e2cb746a9e171f2c492b27c78bb5e433c48f6370f7","afterSHA256":"5ac484a3f720ac4cf71140e2cb746a9e171f2c492b27c78bb5e433c48f6370f7","reason":"Same source body in authored semantic page"},{"oldId":"b-03ba82778dd4a6efb063dd54","targetId":"b-03ba82778dd4a6efb063dd54","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"b5e0f378794cb8b5e98506bac2baa689d3b4fed7d73ae9b28175dfaaf27dbe4b","afterSHA256":"b5e0f378794cb8b5e98506bac2baa689d3b4fed7d73ae9b28175dfaaf27dbe4b","reason":"Same source body in authored semantic page"},{"oldId":"b-5d6fdbb5b0fa0ead2697bd60","targetId":"b-5d6fdbb5b0fa0ead2697bd60","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"3f5dc0bbd2e7534e9bd07a1f682b523e93864d9abfc9188f92d5dc37ff605a01","afterSHA256":"3f5dc0bbd2e7534e9bd07a1f682b523e93864d9abfc9188f92d5dc37ff605a01","reason":"Same source body in authored semantic page"},{"oldId":"b-f6b15b4891f5b52c72bbe384","targetId":"b-f6b15b4891f5b52c72bbe384","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"36259a44332912c3afa0a6cc99679f3f32046cdd68b393af2a095cb0dc8b8542","afterSHA256":"36259a44332912c3afa0a6cc99679f3f32046cdd68b393af2a095cb0dc8b8542","reason":"Same source body in authored semantic page"},{"oldId":"b-1d151f6ddceadca1a1f42124","targetId":"b-1d151f6ddceadca1a1f42124","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"rewrite","beforeSHA256":"c0823b50d032b609150e0475d34c6006c1e48c8a07e376dcdbee74ebc8409eb8","afterSHA256":"762a8c5fbf71524b89e155ad28cc9e70823159c29af9b3f81e9709b17aef7982","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-7a47057f8444dc95ab893627","targetId":"b-7a47057f8444dc95ab893627","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"bdd741431b0bfcf6895a3ec1e163d576a9a39b43e2d32f28568c43e170bf94d1","afterSHA256":"bdd741431b0bfcf6895a3ec1e163d576a9a39b43e2d32f28568c43e170bf94d1","reason":"Same source body in authored semantic page"},{"oldId":"b-b2888cb76ca8131ee51a42c3","targetId":"b-b2888cb76ca8131ee51a42c3","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"63ef714a664c8436ef49a563821c2fb3836648b0e9361da95f69018103d353e6","afterSHA256":"63ef714a664c8436ef49a563821c2fb3836648b0e9361da95f69018103d353e6","reason":"Same source body in authored semantic page"},{"oldId":"b-fc552bd9fb6134b1ab6fe0e9","targetId":"b-fc552bd9fb6134b1ab6fe0e9","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"ba893ef885b664c6949f9713db226dd1b41af3d792fd9ab9f3d46cee131de26b","afterSHA256":"ba893ef885b664c6949f9713db226dd1b41af3d792fd9ab9f3d46cee131de26b","reason":"Same source body in authored semantic page"},{"oldId":"b-b9c2a82f6208826a9de7f4eb","targetId":"b-b9c2a82f6208826a9de7f4eb","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"rewrite","beforeSHA256":"3a586279318b165191ce70d3d763f8cfaa1fde74125422ac404a69584f33b5e0","afterSHA256":"abc9ce8051bfb0d16d084f87dc7c4be7312f5993868cfd50a3846998ba18fc0c","reason":"Same source identity with figure semantics or reading context enriched"},{"oldId":"b-ee146718b861146d72cd029e","targetId":"b-ee146718b861146d72cd029e","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"686b884b78320da1fd4e9f3041bd3fe80f96edebbbe915a529890ae251d76293","afterSHA256":"122ba20c7f0fe242e9c627f9ff6ce34ff9163bdabed0dd9bdccd57346b14e7aa","reason":"Same source body in authored semantic page"},{"oldId":"b-04fd0a2ad4255c9b8da19a8e","targetId":"b-04fd0a2ad4255c9b8da19a8e","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"6d6419fd9b8cf38b3215e50deaae56b50c8735f4332672de31f0b542104500bf","afterSHA256":"6d6419fd9b8cf38b3215e50deaae56b50c8735f4332672de31f0b542104500bf","reason":"Same source body in authored semantic page"},{"oldId":"b-c4d1c85525bf624854d7c0ee","targetId":"b-c4d1c85525bf624854d7c0ee","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"3299792b5c910a1b3a63b2c4677082dda6868df4222b47dc500d9c26add22fe1","afterSHA256":"3299792b5c910a1b3a63b2c4677082dda6868df4222b47dc500d9c26add22fe1","reason":"Same source body in authored semantic page"},{"oldId":"b-58c345b12bdffc9b19b359fb","targetId":"b-58c345b12bdffc9b19b359fb","page":"reading-block-5dc1a98878994b8989105349ef7bd0c6","operation":"move","beforeSHA256":"342160d8a63afceb0d11b164fdae9c202f3e9be639da6a98adb895229b074ebe","afterSHA256":"342160d8a63afceb0d11b164fdae9c202f3e9be639da6a98adb895229b074ebe","reason":"Same source body in authored semantic page"},{"oldId":"b-3f67df864a7d17d3ac873a78","targetId":"b-3f67df864a7d17d3ac873a78","page":"reading-block-de414705e5654ffa86fc46358756877d","operation":"move","beforeSHA256":"04be615138958c4f13e412819a718d3f4afc47be02d77e4b8871d61299bcc9ac","afterSHA256":"04be615138958c4f13e412819a718d3f4afc47be02d77e4b8871d61299bcc9ac","reason":"Same source body in authored semantic page"},{"oldId":"b-c4af94beb4600a179dc01bc8","targetId":"b-c4af94beb4600a179dc01bc8","page":"reading-block-de414705e5654ffa86fc46358756877d","operation":"move","beforeSHA256":"0780b00bec94dbc4e96f95a3bf9900917f0d6e14d81049bff765713475e3bb8a","afterSHA256":"0780b00bec94dbc4e96f95a3bf9900917f0d6e14d81049bff765713475e3bb8a","reason":"Same source body in authored semantic page"},{"oldId":"b-84c4796aeceeaaa99aea2512","targetId":"b-84c4796aeceeaaa99aea2512","page":"reading-block-de414705e5654ffa86fc46358756877d","operation":"move","beforeSHA256":"bc4055cdc99830b8f219631da25dae26736fa6c22ea57660169ddb5b6ac76cde","afterSHA256":"bc4055cdc99830b8f219631da25dae26736fa6c22ea57660169ddb5b6ac76cde","reason":"Same source body in authored semantic page"},{"oldId":"block-a06579c00c0d4cf0a190f71ae8b83559--同样是压缩离散词典与线性子空间留下不同东西","targetId":"b-1e21dee7d58adb4f25797677","operation":"retain-source-identity"},{"oldId":"block-a06579c00c0d4cf0a190f71ae8b83559--pca-为什么可以看成受约束的线性自编码器","targetId":"b-73a8bff2d9f823e3ea9b8437","operation":"retain-source-identity"},{"oldId":"block-5dc1a98878994b8989105349ef7bd0c6--mae-让昂贵编码器只看少量位置","targetId":"b-70d6b80cfe4ac770a53f0b9a","operation":"retain-source-identity"},{"oldId":"block-5dc1a98878994b8989105349ef7bd0c6--已运行的图像重建恢复更好表示评价另算","targetId":"b-b2888cb76ca8131ee51a42c3","operation":"retain-source-identity"},{"oldId":"block-de414705e5654ffa86fc46358756877d--读到这里可以如何继续","targetId":"b-3f67df864a7d17d3ac873a78","operation":"retain-source-identity"}],"blocks":[{"id":"b-35c476a7359c3dc3777adfe0","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-35c476a7359c3dc3777adfe0","content":"# CLS 04 · 重建：约束、位置与表示评价","sha256":"4b32e93b37622319bc268dfcb993d4399abc4ee7dffabf03be88fa420b57d5d0","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-opening","title":"CLS 04 · 重建：约束、位置与表示评价","number":1},"sectionTitle":"CLS 04 · 重建：约束、位置与表示评价","currentQuestion":"CLS 04 · 重建：约束、位置与表示评价","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-3beb81246265a9b6940ad0d4","prerequisites":[{"id":"b-3beb81246265a9b6940ad0d4","url":"#b-3beb81246265a9b6940ad0d4","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3beb81246265a9b6940ad0d4","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-3beb81246265a9b6940ad0d4","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-opening","title":"CLS 04 · 重建：约束、位置与表示评价","number":1},"sectionTitle":"CLS 04 · 重建：约束、位置与表示评价","currentQuestion":"CLS 04 · 重建：约束、位置与表示评价","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-35c476a7359c3dc3777adfe0","nextBlock":"b-122fa7c700d78847699dbeec","prerequisites":[{"id":"b-35c476a7359c3dc3777adfe0","url":"#b-35c476a7359c3dc3777adfe0","role":"section-title"}],"relatedBlocks":[]},{"id":"b-122fa7c700d78847699dbeec","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-122fa7c700d78847699dbeec","content":"词典/PCA数值、线性自编码约束、遮挡索引、masked ConvAE与MAE区别","sha256":"db36e128a2d5a5e8c2d5360cbf60671bab45cb5494bbdd81f27d6b7d95ea74e0","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-opening","title":"CLS 04 · 重建：约束、位置与表示评价","number":1},"sectionTitle":"CLS 04 · 重建：约束、位置与表示评价","currentQuestion":"CLS 04 · 重建：约束、位置与表示评价","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3beb81246265a9b6940ad0d4","nextBlock":"b-169d05e1e9f2b2d606e6f9e2","prerequisites":[{"id":"b-35c476a7359c3dc3777adfe0","url":"#b-35c476a7359c3dc3777adfe0","role":"section-title"},{"id":"b-3beb81246265a9b6940ad0d4","url":"#b-3beb81246265a9b6940ad0d4","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-169d05e1e9f2b2d606e6f9e2","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-169d05e1e9f2b2d606e6f9e2","content":"可先阅读已有结果，再决定是否运行。重算时请核对各节记录的数据、参数、环境与日期。","sha256":"f22152fd55dcdb0090b5653d37145694fd19388d31e10226549464fe8b483578","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-opening","title":"CLS 04 · 重建：约束、位置与表示评价","number":1},"sectionTitle":"CLS 04 · 重建：约束、位置与表示评价","currentQuestion":"CLS 04 · 重建：约束、位置与表示评价","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-122fa7c700d78847699dbeec","nextBlock":null,"prerequisites":[{"id":"b-35c476a7359c3dc3777adfe0","url":"#b-35c476a7359c3dc3777adfe0","role":"section-title"},{"id":"b-3beb81246265a9b6940ad0d4","url":"#b-3beb81246265a9b6940ad0d4","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-2affa291d1e3f6e8ab733866","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-2affa291d1e3f6e8ab733866","content":"**学习路径：** [从遮挡重建到表示评价](/course/study/masked-reconstruction.html)","sha256":"8556351a0a818e5fa2ef8199862458dc0e7165abe7355c0f371bb6d655e593e9","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-b2c6794499589ae3caa596c6","prerequisites":[],"relatedBlocks":[]},{"id":"b-b2c6794499589ae3caa596c6","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-b2c6794499589ae3caa596c6","content":"从[P19—22的MAE位置图](/course/pages/l3-r2-p19.html)进入，本册回答：**限制什么，才能使重建不只是复制输入？**","sha256":"33a9d35da9bf638cbbb8b5e9354a73f8d26efaaa61832a0a1a404bed3367025a","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-2affa291d1e3f6e8ab733866","nextBlock":"b-364cc5857021335d35b1b61d","prerequisites":[{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-364cc5857021335d35b1b61d","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-364cc5857021335d35b1b61d","content":"先读“MAE让昂贵编码器只看少量位置”，手写可见位置、补位顺序和隐藏loss分母；再看[LAB 08](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/b5ba27f2c5ba4ec0922208a7d5dc80a4)保存的同图重建与冻结读出。你的产出是一张对照：模型恢复了哪些像素，哪些下游能力另外测过，两者为什么不能互相代替。","sha256":"2864bf0006511ecd508b2de87e5c323f9679695de8967deee935355762a68f4c","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b2c6794499589ae3caa596c6","nextBlock":"b-c9ea0827b123142b7ef07795","prerequisites":[{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c9ea0827b123142b7ef07795","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-c9ea0827b123142b7ef07795","content":"PCA、去噪与词典的推导保留在前半部，遇到瓶颈或误差问题再回读。需要一个无需神经网络的实际优化入口时，选[LAB 05像素词典](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/de39b998077f44cca4b8407085547766)。可直接引用[隐藏loss分母](/course/blocks/mae-masked-loss-denominator.html)或[重建限制](/course/blocks/reconstruction-restrictions.html)。","sha256":"9e07fc295fe95877d90b697ad5fa48296f312a7ab8d0889774a5f486fd8d090d","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-364cc5857021335d35b1b61d","nextBlock":"b-b7e8ad649ebc19896d40d215","prerequisites":[{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b7e8ad649ebc19896d40d215","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-b7e8ad649ebc19896d40d215","content":"如果不提供“猫”“杯子”这样的类别标签，图像还有什么可以当答案？最直接的是图像自身：把它压缩、破坏或遮住一部分，再让模型恢复。困难不在于是否有监督数值，而在于我们允许模型怎样取得它。把输入原样复制到输出可以获得零误差，却没有迫使模型形成有用的表示。重建方法之间真正改变的是限制条件：代码可以取什么值、信息从哪里通过、哪些位置可见，以及误差计算在哪里。","sha256":"a8be541fcaf19bfce2bf41f6779590be20eb214a5f4faf49be22dcf9a7d9bbc0","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c9ea0827b123142b7ef07795","nextBlock":"b-1e21dee7d58adb4f25797677","prerequisites":[{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-1e21dee7d58adb4f25797677","type":"h2","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-1e21dee7d58adb4f25797677","content":"## 同样是压缩，离散词典与线性子空间留下不同东西","sha256":"9a8088a16b7096a914d6ede16623634ce17a57528701ccdb6e9cb90e6d1c8e7f","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b7e8ad649ebc19896d40d215","nextBlock":"b-f2c2667742ed6f6fef0e5d65","prerequisites":[{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-f2c2667742ed6f6fef0e5d65","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-f2c2667742ed6f6fef0e5d65","content":"把一个 $2\\times2$ 灰度 patch 展平为 $x\\in\\mathbb R^4$。K-means 学习 $K$ 个中心 $c_k$，用最近中心的索引编码：","sha256":"ab7a10e9728afbecdcec79d3f12505658449566b8277e7632a8f41cdc23677f7","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"2\\times2","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"x\\in\\mathbb R^4","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"K","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"c_k","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-1e21dee7d58adb4f25797677","nextBlock":"b-578e14dc3bb335659b9dedfa","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-578e14dc3bb335659b9dedfa","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-578e14dc3bb335659b9dedfa","content":"$a(x)=\\arg\\min_k\\|x-c_k\\|^2,\\qquad \\hat x=c_{a(x)}.$","sha256":"aef13a0897067ed8c716ad7ba8fabf84019f69a5ea2017415804af017d79c7b6","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"a(x)=\\arg\\min_k\\|x-c_k\\|^2,\\qquad \\hat x=c_{a(x)}.","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-f2c2667742ed6f6fef0e5d65","nextBlock":"b-b661e33088f8d4d3a9e3e347","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b661e33088f8d4d3a9e3e347","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-b661e33088f8d4d3a9e3e347","content":"若词典恰为 $c_1=(1,0,1,0)$、$c_2=(0,1,0,1)$，它可以零误差重建左右两种明暗边，但 $(0.5,0.5,0.5,0.5)$ 到两中心的平方距离都为 1，只能按约定任选其一。代码被限制为一个离散索引，约需 $\\log_2 K$ bit，另须存储词典；这不是任意实数向量的无损压缩。改变 $K$ 会改变量化误差与存储成本。原 [LAB 05](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/de39b998077f44cca4b8407085547766?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=de39b998077f44cca4b8407085547766&utm_term=get_notebook) 在真实 MNIST patch 上进行 Lloyd 拟合，可查看中心怎样随数据形成，而非只背“无监督”这个标签。","sha256":"b95322b58e86aab97b9239ea97f9fbcd76a279b78c510fe88e115594dd0cd9ea","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"c_1=(1,0,1,0)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"c_2=(0,1,0,1)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"(0.5,0.5,0.5,0.5)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"\\log_2 K","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"K","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-578e14dc3bb335659b9dedfa","nextBlock":"b-e2a332133a492c1bf17355e3","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-e2a332133a492c1bf17355e3","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-e2a332133a492c1bf17355e3","content":"PCA 改用连续线性坐标。对已减去均值的 $x$，选列正交矩阵 $U\\in\\mathbb R^{d\\times k}$，编码 $z=U^Tx$，重建 $\\hat x=Uz$。最小化平方重建误差等价于保留协方差最大的 $k$ 个方向。","sha256":"314d138cb51e798d2754f0d3a4ea9e9f5db2ea2051da32427b62b50db0088186","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"x","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"U\\in\\mathbb R^{d\\times k}","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"z=U^Tx","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"\\hat x=Uz","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"k","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-b661e33088f8d4d3a9e3e347","nextBlock":"b-ee6425c0f1570c24b5dc8797","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ee6425c0f1570c24b5dc8797","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-ee6425c0f1570c24b5dc8797","content":"用四点 $(2,0),(-2,0),(0,1),(0,-1)$ 手算。均值为 0，按 $1/4$ 计算协方差是 $\\mathrm{diag}(2,0.5)$。只保留一维时选横轴，四点重建为 $(2,0),(-2,0),(0,0),(0,0)$；逐点平方误差为 $0,0,1,1$，平均 0.5。若选纵轴，平均误差为 2。被丢掉的方向方差小，不代表它对任务不重要：如果标签恰好区分纵坐标正负，PCA 的最优重建轴可能删去标签所需的差异。","sha256":"da8a5c96a3492ad378af6c0cbf696fd54ccd53c2c29bbc8e794d30c3a07e225e","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"(2,0),(-2,0),(0,1),(0,-1)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"1/4","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"\\mathrm{diag}(2,0.5)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"(2,0),(-2,0),(0,0),(0,0)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"0,0,1,1","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-e2a332133a492c1bf17355e3","nextBlock":"b-34bb8503809a97fdc5af70e9","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-34bb8503809a97fdc5af70e9","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-34bb8503809a97fdc5af70e9","content":"![PCA 的平方误差与 MAE 的位置恢复](/course/repair/assets/cls05-pca-mask.svg)","sha256":"5e93b74f70085ca5ec3b76b71d57b2a105f5a18720df2082b57f0ac8f1df1499","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ee6425c0f1570c24b5dc8797","nextBlock":"b-684445c71410a0f198f8b4fc","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-684445c71410a0f198f8b4fc","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-684445c71410a0f198f8b4fc","content":"左图按真实坐标画出四点和一维投影；右图展示 patch 的乱序、可见项与恢复位置。两者都在重建，但一个限制子空间，一个限制输入可见性。","sha256":"7ed392f7e5f8093e27fad86ce7c3c1dd94875138594c09c1db623b85a4946a51","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-34bb8503809a97fdc5af70e9","nextBlock":"b-73a8bff2d9f823e3ea9b8437","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-73a8bff2d9f823e3ea9b8437","type":"h2","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-73a8bff2d9f823e3ea9b8437","content":"## PCA 为什么可以看成受约束的线性自编码器","sha256":"859cfeee5fb3ba73e84cd3892eaf1ea08e08ffef5ebf530789c7027945093b85","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-684445c71410a0f198f8b4fc","nextBlock":"b-8ebb2b48c62b2d2cc69790d1","prerequisites":[{"id":"b-1e21dee7d58adb4f25797677","url":"#b-1e21dee7d58adb4f25797677","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-8ebb2b48c62b2d2cc69790d1","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-8ebb2b48c62b2d2cc69790d1","content":"自编码器由 $z=f_\\theta(x)$ 与 $\\hat x=g_\\phi(z)$ 组成，以 $\\|x-\\hat x\\|^2$ 等重建损失更新参数。令 $f(x)=U^Tx,g(z)=Uz$ 并要求 $U^TU=I$，就得到上面的 PCA 形式。一般线性、低秩、平方误差自编码器也能学到 PCA 的主子空间，尽管编码坐标不一定等于标准主成分；平移、旋转等参数化差异要与重建子空间区分。","sha256":"1ec743964fe5a8b6a41542c2f230348bcf69b21f317b8a22ea2f1483ada27a4b","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"z=f_\\theta(x)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"\\hat x=g_\\phi(z)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"\\|x-\\hat x\\|^2","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"f(x)=U^Tx,g(z)=Uz","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"U^TU=I","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-73a8bff2d9f823e3ea9b8437","nextBlock":"b-64a3661efdaa684b3fc9aaf2","prerequisites":[{"id":"b-73a8bff2d9f823e3ea9b8437","url":"#b-73a8bff2d9f823e3ea9b8437","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-64a3661efdaa684b3fc9aaf2","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-64a3661efdaa684b3fc9aaf2","content":"所以 SVD 可以精确说明一种线性自编码约束，但 SVD 本身没有执行一个另行承诺的非线性网络训练。引入非线性编码器和解码器以后，表示可以适应弯曲的数据结构；也可能凭较大容量记忆训练样本。瓶颈维数、正则化、跳连是否绕过瓶颈、训练与保留误差，都影响我们能否把低损失解释为学到了结构。","sha256":"e62fcdbdf8f5d621235c6d665a17152c1736f9eb0ff056daa0c2d302d56ee703","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-8ebb2b48c62b2d2cc69790d1","nextBlock":"b-3c6fcb459cbbbf5fa64a3c4e","prerequisites":[{"id":"b-73a8bff2d9f823e3ea9b8437","url":"#b-73a8bff2d9f823e3ea9b8437","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3c6fcb459cbbbf5fa64a3c4e","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-3c6fcb459cbbbf5fa64a3c4e","content":"去噪自编码器进一步改变训练对：先从破坏分布采样 $\\tilde x\\sim q(\\tilde x\\mid x)$，再最小化 $\\mathbb E\\|g(f(\\tilde x))-x\\|^2$。输入是带噪图片，目标仍是干净图片。训练时不给模型干净图，不能在可视化时偷偷把目标的边缘或 mask 喂回去。Gaussian 噪声、置零像素和随机遮块提出不同恢复任务；噪声太轻会接近复制，太重可能留下不足以判断的证据。在 [去噪自编码器原论文](https://www.jmlr.org/papers/v11/vincent10a.html) 中，破坏过程正是组织表示学习的一个关键约束。","sha256":"4d82f52f7518bc4d90a45a49b8188143486de499bd9a390f4e9be7d7b2985f15","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"\\tilde x\\sim q(\\tilde x\\mid x)","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"},{"latex":"\\mathbb E\\|g(f(\\tilde x))-x\\|^2","definitionContext":"同样是压缩，离散词典与线性子空间留下不同东西"}],"previousBlock":"b-64a3661efdaa684b3fc9aaf2","nextBlock":"b-d64fd917b95d2f2d11b93a45","prerequisites":[{"id":"b-73a8bff2d9f823e3ea9b8437","url":"#b-73a8bff2d9f823e3ea9b8437","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d64fd917b95d2f2d11b93a45","type":"p","scope":"block-a06579c00c0d4cf0a190f71ae8b83559","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-d64fd917b95d2f2d11b93a45","content":"平方误差还有一个值得记住的性质：最佳预测是给定输入后的条件均值。若一个被遮挡像素在其余条件相同下等概率为 0 或 1，输出 0.5 的平均平方误差是 0.25，而总是输出 0 或 1 的误差是 0.5。模糊有时来自目标的统计偏好，不一定只是网络没有优化好。下一讲讨论采样时会重新遇到这个问题。","sha256":"54a1680c2cde89592947009c399bb9696954ae9ee53a7738e72503f17248e65a","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-a06579c00c0d4cf0a190f71ae8b83559","title":"同样是压缩，离散词典与线性子空间留下不同东西","number":2},"sectionTitle":"同样是压缩，离散词典与线性子空间留下不同东西","currentQuestion":"同样是压缩，离散词典与线性子空间留下不同东西","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3c6fcb459cbbbf5fa64a3c4e","nextBlock":null,"prerequisites":[{"id":"b-73a8bff2d9f823e3ea9b8437","url":"#b-73a8bff2d9f823e3ea9b8437","role":"section-title"},{"id":"b-2affa291d1e3f6e8ab733866","url":"#b-2affa291d1e3f6e8ab733866","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-70d6b80cfe4ac770a53f0b9a","type":"h2","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-70d6b80cfe4ac770a53f0b9a","content":"## MAE 让昂贵编码器只看少量位置","sha256":"94c92248638a95e30f63c78bc247c7bac3e334d71bda920fd01554a003dbda6d","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-cb1376ea688d9d381bb3f36c","prerequisites":[{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-cb1376ea688d9d381bb3f36c","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-cb1376ea688d9d381bb3f36c","content":"把图像切为 patch 后，MAE 随机选择可见子集，编码器只处理可见 patch 及其位置；轻量解码器补入 mask token，恢复完整位置顺序，再预测被遮挡的像素目标。原 MAE 的一个关键设计是高遮挡率与这种不对称编码—解码结构，见 [MAE §3 与图 1](https://arxiv.org/html/2111.06377v3)。这里不是先让编码器看完整图，再在输出端涂黑几块。","sha256":"11689a6e9d1b70e1f58e6c9dc369e5b3e87fa0cb70c4c6f28d85cbea174eb60a","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-70d6b80cfe4ac770a53f0b9a","nextBlock":"b-61d52de149887e22ba65ab3e","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"}],"relatedBlocks":[]},{"id":"b-61d52de149887e22ba65ab3e","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-61d52de149887e22ba65ab3e","content":"四个原始位置是 `[0,1,2,3]`。随机排列 `ids_shuffle=[2,0,3,1]`，只保留前两项，所以编码器看到原位置 2、0。解码器先得到 `[z2,z0,M,M]`，再用逆排列 `ids_restore=[1,3,0,2]` 取回 `[z0,M,z2,M]`。位置 1、3 的损失可以按","sha256":"5ac484a3f720ac4cf71140e2cb746a9e171f2c492b27c78bb5e433c48f6370f7","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-cb1376ea688d9d381bb3f36c","nextBlock":"b-03ba82778dd4a6efb063dd54","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-03ba82778dd4a6efb063dd54","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-03ba82778dd4a6efb063dd54","content":"$L=\\frac{1}{|\\mathcal M|}\\sum_{i\\in\\mathcal M}\\|\\hat x_i-x_i\\|^2$","sha256":"b5e0f378794cb8b5e98506bac2baa689d3b4fed7d73ae9b28175dfaaf27dbe4b","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"L=\\frac{1}{|\\mathcal M|}\\sum_{i\\in\\mathcal M}\\|\\hat x_i-x_i\\|^2","definitionContext":"MAE 让昂贵编码器只看少量位置"}],"previousBlock":"b-61d52de149887e22ba65ab3e","nextBlock":"b-5d6fdbb5b0fa0ead2697bd60","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-5d6fdbb5b0fa0ead2697bd60","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-5d6fdbb5b0fa0ead2697bd60","content":"计算，其中 $\\mathcal M$ 为遮挡位置集合。若补 mask 后忘记恢复顺序，即便预测数值合理，也是在与错误位置的目标比较。排列练习可以证明索引正确；它不能证明编码器已经学会图像语义。","sha256":"3f5dc0bbd2e7534e9bd07a1f682b523e93864d9abfc9188f92d5dc37ff605a01","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"\\mathcal M","definitionContext":"MAE 让昂贵编码器只看少量位置"}],"previousBlock":"b-03ba82778dd4a6efb063dd54","nextBlock":"b-f6b15b4891f5b52c72bbe384","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-f6b15b4891f5b52c72bbe384","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-f6b15b4891f5b52c72bbe384","content":"再看计算：16 个 patch 只保留 4 个，编码器注意力的成对位置数由 $16^2=256$ 变为 $4^2=16$，这是二次项的 1/16；线性层、patch embedding、解码器和实际硬件开销不按同样比例缩小。因此不能把这个比例直接称为端到端加速 16 倍。高 mask 比例同时改变任务难度与编码成本，需要一起讨论。","sha256":"36259a44332912c3afa0a6cc99679f3f32046cdd68b393af2a095cb0dc8b8542","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[{"latex":"16^2=256","definitionContext":"MAE 让昂贵编码器只看少量位置"},{"latex":"4^2=16","definitionContext":"MAE 让昂贵编码器只看少量位置"}],"previousBlock":"b-5d6fdbb5b0fa0ead2697bd60","nextBlock":"b-1d151f6ddceadca1a1f42124","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-1d151f6ddceadca1a1f42124","type":"table","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-1d151f6ddceadca1a1f42124","content":"| 方法 | 输入受什么限制 | 代码/表示约束 | 重建好能支持什么判断 |\n| --- | --- | --- | --- |\n| K-means 词典 | 通常看到完整 patch | 一个中心索引 | 该词典的量化误差小 |\n| PCA | 通常看到完整向量 | 低维线性子空间 | 方差被线性子空间有效保留 |\n| 去噪自编码器 | 看到明确破坏后的图 | 架构、瓶颈及正则化 | 对指定破坏能恢复保留图像 |\n| MAE | 编码器只看可见 patch | 可见位置及编码—解码设计 | 能利用上下文预测遮挡目标 |","sha256":"762a8c5fbf71524b89e155ad28cc9e70823159c29af9b3f81e9709b17aef7982","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["方法","输入受什么限制","代码/表示约束","重建好能支持什么判断"],"rows":[["方法","输入受什么限制","代码/表示约束","重建好能支持什么判断"],["K-means 词典","通常看到完整 patch","一个中心索引","该词典的量化误差小"],["PCA","通常看到完整向量","低维线性子空间","方差被线性子空间有效保留"],["去噪自编码器","看到明确破坏后的图","架构、瓶颈及正则化","对指定破坏能恢复保留图像"],["MAE","编码器只看可见 patch","可见位置及编码—解码设计","能利用上下文预测遮挡目标"]]},"previousBlock":"b-f6b15b4891f5b52c72bbe384","nextBlock":"b-7a47057f8444dc95ab893627","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7a47057f8444dc95ab893627","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-7a47057f8444dc95ab893627","content":"这四行都没有自动回答“少量标签能否读出类别”“颜色是否保留”“边界是否准确”。那些是另设的评价问题。[LAB 08](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/b5ba27f2c5ba4ec0922208a7d5dc80a4) 把索引检查与图像重建分开：先确认输入—mask—目标位置，再比较输入、重建和误差，最后冻结编码器作读出。划分必须先按原图进行，再生成破坏版本；否则同一原图的干净版和带噪版可能分居训练与测试，给出过于乐观的结果。","sha256":"bdd741431b0bfcf6895a3ec1e163d576a9a39b43e2d32f28568c43e170bf94d1","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-1d151f6ddceadca1a1f42124","nextBlock":"b-b2888cb76ca8131ee51a42c3","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b2888cb76ca8131ee51a42c3","type":"h3","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-b2888cb76ca8131ee51a42c3","content":"### 已运行的图像重建：恢复更好，表示评价另算","sha256":"63ef714a664c8436ef49a563821c2fb3836648b0e9361da95f69018103d353e6","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7a47057f8444dc95ab893627","nextBlock":"b-fc552bd9fb6134b1ab6fe0e9","prerequisites":[{"id":"b-70d6b80cfe4ac770a53f0b9a","url":"#b-70d6b80cfe4ac770a53f0b9a","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-fc552bd9fb6134b1ab6fe0e9","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-fc552bd9fb6134b1ab6fe0e9","content":"LAB 08 在 2026-09-22 本地 CPU 从头训练卷积自编码器。数据是 `sklearn load_digits` 的 1797 张 8×8 灰度图，先按原图分层随机划分 1257 训练、540 保留，再生成破坏；没有按书写者分组，因此结论是这个原图随机划分下的数字任务。标签只用于划分与读出，不进入重建损失。16 维瓶颈的去噪网络训练 40 epoch，破坏为 25% 像素丢弃加标准差 0.15 的高斯噪声并裁到 [0,1]。","sha256":"ba893ef885b664c6949f9713db226dd1b41af3d792fd9ab9f3d46cee131de26b","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b2888cb76ca8131ee51a42c3","nextBlock":"b-b9c2a82f6208826a9de7f4eb","prerequisites":[{"id":"b-b2888cb76ca8131ee51a42c3","url":"#b-b2888cb76ca8131ee51a42c3","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-b9c2a82f6208826a9de7f4eb","type":"table","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-b9c2a82f6208826a9de7f4eb","content":"| 保留集方法 | 与干净目标的逐像素 MSE |\n| --- | --- |\n| 直接使用带噪输入 | 0.06244 |\n| 训练集拟合的 rank-16 PCA 重建 | 0.04160 |\n| 学到的卷积去噪自编码器 | 0.03217 |","sha256":"abc9ce8051bfb0d16d084f87dc7c4be7312f5993868cfd50a3846998ba18fc0c","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"table":{"headers":["保留集方法","与干净目标的逐像素 MSE"],"rows":[["保留集方法","与干净目标的逐像素 MSE"],["直接使用带噪输入","0.06244"],["训练集拟合的 rank-16 PCA 重建","0.04160"],["学到的卷积去噪自编码器","0.03217"]]},"previousBlock":"b-fc552bd9fb6134b1ab6fe0e9","nextBlock":"b-ee146718b861146d72cd029e","prerequisites":[{"id":"b-b2888cb76ca8131ee51a42c3","url":"#b-b2888cb76ca8131ee51a42c3","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-ee146718b861146d72cd029e","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-ee146718b861146d72cd029e","content":"![真实输入、破坏、重建与遮挡恢复](/assets/display/cdcfe2b4d2103d5484256b71.webp)","sha256":"122ba20c7f0fe242e9c627f9ff6ce34ff9163bdabed0dd9bdccd57346b14e7aa","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b9c2a82f6208826a9de7f4eb","nextBlock":"b-04fd0a2ad4255c9b8da19a8e","prerequisites":[{"id":"b-b2888cb76ca8131ee51a42c3","url":"#b-b2888cb76ca8131ee51a42c3","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-04fd0a2ad4255c9b8da19a8e","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-04fd0a2ad4255c9b8da19a8e","content":"另一个两通道卷积模型接收“被遮挡图像＋可见性 mask”，在 50% 的 2×2 patch 隐藏率下训练 50 epoch。它处理整张密集网格，**不是原论文只编码可见 token 的 ViT MAE**。固定每图同一排列、让隐藏集合嵌套，测试 25%/50%/75% 遮挡时，隐藏位置 MSE 分别为 0.03933/0.04341/0.05563；相应训练均值基线为 0.07420/0.07402/0.07390。遮挡增加后重建变难，但这个卷积模型并未获得 MAE 的可见 token 计算节省。","sha256":"6d6419fd9b8cf38b3215e50deaae56b50c8735f4332672de31f0b542104500bf","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-ee146718b861146d72cd029e","nextBlock":"b-c4d1c85525bf624854d7c0ee","prerequisites":[{"id":"b-b2888cb76ca8131ee51a42c3","url":"#b-b2888cb76ca8131ee51a42c3","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c4d1c85525bf624854d7c0ee","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-c4d1c85525bf624854d7c0ee","content":"冻结编码器、输入干净保留图像作 5NN，去噪架构由随机初始化的 84.26% 到训练后的 95.00%；遮挡架构由 88.52% 到 92.04%。两种架构输入通道不同，随机基线也应分别保留，不能把它们混成同一个起点。重建改善和读出改善在这次实验中同时出现，但它们仍是两项独立测量。配置、输入 hash 与权重均在 LAB 08 对应材料中。","sha256":"3299792b5c910a1b3a63b2c4677082dda6868df4222b47dc500d9c26add22fe1","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-04fd0a2ad4255c9b8da19a8e","nextBlock":"b-58c345b12bdffc9b19b359fb","prerequisites":[{"id":"b-b2888cb76ca8131ee51a42c3","url":"#b-b2888cb76ca8131ee51a42c3","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-58c345b12bdffc9b19b359fb","type":"p","scope":"block-5dc1a98878994b8989105349ef7bd0c6","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-58c345b12bdffc9b19b359fb","content":"重建为“保存信息”提供了可计算目标；下一册 [CLS 05](https://deepnote.com/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/b8320404134f4a2e858ccfe7d50a71d3) 会主动要求某些变化在表示中不产生差异。二者的张力是具体的：准确重建颜色需要留下颜色，颜色不变的对比目标却可能鼓励忽略它。好的表示要服务什么，必须由下游问题回答。","sha256":"342160d8a63afceb0d11b164fdae9c202f3e9be639da6a98adb895229b074ebe","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-5dc1a98878994b8989105349ef7bd0c6","title":"MAE 让昂贵编码器只看少量位置","number":3},"sectionTitle":"MAE 让昂贵编码器只看少量位置","currentQuestion":"MAE 让昂贵编码器只看少量位置","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c4d1c85525bf624854d7c0ee","nextBlock":null,"prerequisites":[{"id":"b-b2888cb76ca8131ee51a42c3","url":"#b-b2888cb76ca8131ee51a42c3","role":"section-title"},{"id":"b-cb1376ea688d9d381bb3f36c","url":"#b-cb1376ea688d9d381bb3f36c","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-3f67df864a7d17d3ac873a78","type":"h3","scope":"block-de414705e5654ffa86fc46358756877d","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-3f67df864a7d17d3ac873a78","content":"### 读到这里，可以如何继续？","sha256":"04be615138958c4f13e412819a718d3f4afc47be02d77e4b8871d61299bcc9ac","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-de414705e5654ffa86fc46358756877d","title":"读到这里，可以如何继续？","number":4},"sectionTitle":"读到这里，可以如何继续？","currentQuestion":"读到这里，可以如何继续？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-c4af94beb4600a179dc01bc8","prerequisites":[{"id":"b-c4af94beb4600a179dc01bc8","url":"#b-c4af94beb4600a179dc01bc8","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-c4af94beb4600a179dc01bc8","type":"p","scope":"block-de414705e5654ffa86fc46358756877d","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-c4af94beb4600a179dc01bc8","content":"若你能解释“限制输入”与“限制表示”有什么不同，并能给16块示意写出隐藏loss分母，就已完成本册机制主线。接下来选 [LAB 08 · 图像重建：遮挡损失与表示评价](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/b5ba27f2c5ba4ec0922208a7d5dc80a4?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=b5ba27f2c5ba4ec0922208a7d5dc80a4&utm_term=get_notebook)：先看真实输入、重建、误差，最后读冻结5NN。其模型是masked ConvAE，仍处理稠密网格；原MAE的可见token节省计算由 [少看图块为何省计算，又为何不是固定加速比](/course/blocks/mae-compute-not-sixteen-times.html) 单独解释。","sha256":"0780b00bec94dbc4e96f95a3bf9900917f0d6e14d81049bff765713475e3bb8a","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-de414705e5654ffa86fc46358756877d","title":"读到这里，可以如何继续？","number":4},"sectionTitle":"读到这里，可以如何继续？","currentQuestion":"读到这里，可以如何继续？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-3f67df864a7d17d3ac873a78","nextBlock":"b-84c4796aeceeaaa99aea2512","prerequisites":[{"id":"b-3f67df864a7d17d3ac873a78","url":"#b-3f67df864a7d17d3ac873a78","role":"section-title"}],"relatedBlocks":[]},{"id":"b-84c4796aeceeaaa99aea2512","type":"p","scope":"block-de414705e5654ffa86fc46358756877d","url":"https://codingai-lec03.pages.dev/course/cls05-重建-词典pca自编码器与-mae.html#b-84c4796aeceeaaa99aea2512","content":"只想看一个无需神经网络的实际无标签优化，可选 [LAB 05 · 像素 patch 词典：拟合与重建](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/de39b998077f44cca4b8407085547766?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=de39b998077f44cca4b8407085547766&utm_term=get_notebook) 的MNIST patch词典。关于教师目标的旧拓展已由 [CLS 06 · 表示的三个检查：目标、信息与新域](https://deepnote.com/workspace/ise3309jingao-5ba66b0a-4283-4f5a-ae06-3f68d3201e99/project/38755e4e-23f6-4f5f-8c68-8a2e4114b1c1/notebook/c0645a4db6e04188ab5b119e87c53bed?utm_source=openai&utm_medium=mcp&utm_campaign=openaimcp&utm_content=c0645a4db6e04188ab5b119e87c53bed&utm_term=get_notebook) 统一组织，不需要在词典实验末尾再学一遍。最后回到 [P23 本页说明](/course/pages/l3-r2-p23.html)：重建好与新任务好，需要分别测量。","sha256":"bc4055cdc99830b8f219631da25dae26736fa6c22ea57660169ddb5b6ac76cde","chapter":{"id":"course","title":"CLS 04 · 重建：约束、位置与表示评价"},"unit":{"id":"cls05-重建-词典pca自编码器与-mae","title":"CLS 04 · 重建：约束、位置与表示评价"},"page":{"id":"reading-block-de414705e5654ffa86fc46358756877d","title":"读到这里，可以如何继续？","number":4},"sectionTitle":"读到这里，可以如何继续？","currentQuestion":"读到这里，可以如何继续？","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-c4af94beb4600a179dc01bc8","nextBlock":null,"prerequisites":[{"id":"b-3f67df864a7d17d3ac873a78","url":"#b-3f67df864a7d17d3ac873a78","role":"section-title"},{"id":"b-c4af94beb4600a179dc01bc8","url":"#b-c4af94beb4600a179dc01bc8","role":"problem-context"}],"relatedBlocks":[]}]}