{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.md","pages":[{"id":"reading-opening","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","unit":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","unitId":"notebook-blocks/645d32ebb74d4ed68f805e40027fb013","chapter":"course","number":1}],"aliases":[{"oldId":"b-757367f2cc58a76444f4e1d9","targetId":"b-757367f2cc58a76444f4e1d9","page":"reading-opening","operation":"move","beforeSHA256":"1a4de4df51c1a8787dc96ec401888dfe89d250842519319b9af7a931ca765eee","afterSHA256":"1a4de4df51c1a8787dc96ec401888dfe89d250842519319b9af7a931ca765eee","reason":"Same source body in authored semantic page"},{"oldId":"b-44c146a062a22c6de0911722","targetId":"b-44c146a062a22c6de0911722","page":"reading-opening","operation":"move","beforeSHA256":"1de034ec9c721d9474a1301daf10b8c8c97626f8f7022cd82ab26317d9c3df6d","afterSHA256":"1de034ec9c721d9474a1301daf10b8c8c97626f8f7022cd82ab26317d9c3df6d","reason":"Same source body in authored semantic page"},{"oldId":"b-d7009bb7cc829d7a492ff3a5","targetId":"b-d7009bb7cc829d7a492ff3a5","page":"reading-opening","operation":"move","beforeSHA256":"dc06c1e728a4b5740ba634b145bc9a1da6323031626734cb4555db5a744df806","afterSHA256":"dc06c1e728a4b5740ba634b145bc9a1da6323031626734cb4555db5a744df806","reason":"Same source body in authored semantic page"},{"oldId":"b-0f54f4932fd6f935a31c947c","targetId":"b-0f54f4932fd6f935a31c947c","page":"reading-opening","operation":"move","beforeSHA256":"8021c18afc52ba7b6612f43ecfb0fdd4789efe91cc9cd656984357d81a8514de","afterSHA256":"8021c18afc52ba7b6612f43ecfb0fdd4789efe91cc9cd656984357d81a8514de","reason":"Same source body in authored semantic page"},{"oldId":"b-72eae900e12963709941571c","targetId":"b-72eae900e12963709941571c","page":"reading-opening","operation":"move","beforeSHA256":"03548e94c27e646b9073eeed385278bb117959dca4e57b428eeb514af543181c","afterSHA256":"03548e94c27e646b9073eeed385278bb117959dca4e57b428eeb514af543181c","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-757367f2cc58a76444f4e1d9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html#b-757367f2cc58a76444f4e1d9","content":"[LAB 12 · CLIP延伸：缓存与检索（近似索引选读）](/course/notebooks/921eeaf784284ce19568c6ca7619b55a.html)","sha256":"1a4de4df51c1a8787dc96ec401888dfe89d250842519319b9af7a931ca765eee","chapter":{"id":"course","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"unit":{"id":"notebook-blocks/645d32ebb74d4ed68f805e40027fb013","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"page":{"id":"reading-opening","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","number":1},"sectionTitle":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","currentQuestion":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-44c146a062a22c6de0911722","prerequisites":[],"relatedBlocks":[]},{"id":"b-44c146a062a22c6de0911722","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html#b-44c146a062a22c6de0911722","content":"# 3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","sha256":"1de034ec9c721d9474a1301daf10b8c8c97626f8f7022cd82ab26317d9c3df6d","chapter":{"id":"course","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"unit":{"id":"notebook-blocks/645d32ebb74d4ed68f805e40027fb013","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"page":{"id":"reading-opening","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","number":1},"sectionTitle":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","currentQuestion":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-757367f2cc58a76444f4e1d9","nextBlock":"b-d7009bb7cc829d7a492ff3a5","prerequisites":[{"id":"b-757367f2cc58a76444f4e1d9","url":"#b-757367f2cc58a76444f4e1d9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-d7009bb7cc829d7a492ff3a5","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html#b-d7009bb7cc829d7a492ff3a5","content":"并入相关学习路径","sha256":"dc06c1e728a4b5740ba634b145bc9a1da6323031626734cb4555db5a744df806","chapter":{"id":"course","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"unit":{"id":"notebook-blocks/645d32ebb74d4ed68f805e40027fb013","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"page":{"id":"reading-opening","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","number":1},"sectionTitle":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","currentQuestion":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-44c146a062a22c6de0911722","nextBlock":"b-0f54f4932fd6f935a31c947c","prerequisites":[{"id":"b-44c146a062a22c6de0911722","url":"#b-44c146a062a22c6de0911722","role":"section-title"},{"id":"b-757367f2cc58a76444f4e1d9","url":"#b-757367f2cc58a76444f4e1d9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-0f54f4932fd6f935a31c947c","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html#b-0f54f4932fd6f935a31c947c","content":"同一CLIP缓存、exact/IVF搜索、延迟/扫描量、区域交互基线","sha256":"8021c18afc52ba7b6612f43ecfb0fdd4789efe91cc9cd656984357d81a8514de","chapter":{"id":"course","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"unit":{"id":"notebook-blocks/645d32ebb74d4ed68f805e40027fb013","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"page":{"id":"reading-opening","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","number":1},"sectionTitle":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","currentQuestion":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d7009bb7cc829d7a492ff3a5","nextBlock":"b-72eae900e12963709941571c","prerequisites":[{"id":"b-44c146a062a22c6de0911722","url":"#b-44c146a062a22c6de0911722","role":"section-title"},{"id":"b-757367f2cc58a76444f4e1d9","url":"#b-757367f2cc58a76444f4e1d9","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-72eae900e12963709941571c","type":"pre","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/645d32ebb74d4ed68f805e40027fb013.html#b-72eae900e12963709941571c","content":"```\nunit=cr.unit\ndef spherical_kmeans(X,nlist=8,seed=17,iterations=40):\n    rng=np.random.default_rng(seed)\n    centers=X[rng.choice(len(X),nlist,replace=False)].copy()\n    for _ in range(iterations):\n        ids=(X@centers.T).argmax(1)\n        new=np.stack([unit(X[ids==i].mean(0,keepdims=True))[0] if np.any(ids==i) else centers[i] for i in range(nlist)])\n        if np.allclose(new,centers,atol=1e-7):break\n        centers=new\n    ids=(X@centers.T).argmax(1)\n    return centers,[np.flatnonzero(ids==i) for i in range(nlist)]\n\ncenters,lists=spherical_kmeans(X,nlist=8,seed=17)\ndef ivf_search(q,nprobe,k=3):\n    buckets=cr.stable_rank(q@centers.T,k=nprobe)\n    candidates=np.concatenate([lists[int(b)] for b in buckets])\n    ranked=cr.stable_rank(q@X[candidates].T,ids=candidates,k=k)\n    return ranked,candidates\nfor nprobe in [1,2,4,8]:\n    answer=[ivf_search(q,nprobe) for q in Q]\n    overlap=np.mean([len(set(e)&set(r))/3 for e,(r,c) in zip(exact,answer)])\n    hit=np.mean([any(labels[j]==truth[i] for j in r) for i,(r,c) in enumerate(answer)])\n    print('nprobe',nprobe,'mean scanned',np.mean([len(c) for r,c in answer]),'overlap',overlap,'Hit@3',hit)\n    if nprobe==8:assert all(np.array_equal(e,r) for e,(r,c) in zip(exact,answer))\n# 逐项找出 nprobe=1 的遗漏，不只看一个平均分。\nfor i,q in enumerate(Q):\n    ranked,candidates=ivf_search(q,1)\n    missing=sorted(set(exact[i])-set(ranked))\n    if missing:\n        print('query:',manifest['queries'][i]['text'])\n        print('missed exact neighbors:',[manifest['gallery'][j]['id'] for j in missing])\n\n```","sha256":"03548e94c27e646b9073eeed385278bb117959dca4e57b428eeb514af543181c","chapter":{"id":"course","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"unit":{"id":"notebook-blocks/645d32ebb74d4ed68f805e40027fb013","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码"},"page":{"id":"reading-opening","title":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","number":1},"sectionTitle":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","currentQuestion":"3 · 实际的近似索引：先路由到簇，再在簇内精排 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-0f54f4932fd6f935a31c947c","nextBlock":null,"prerequisites":[{"id":"b-44c146a062a22c6de0911722","url":"#b-44c146a062a22c6de0911722","role":"section-title"},{"id":"b-757367f2cc58a76444f4e1d9","url":"#b-757367f2cc58a76444f4e1d9","role":"problem-context"}],"relatedBlocks":[]}]}