{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"冻结评价回答另一个问题 · 实现代码","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.md","pages":[{"id":"reading-opening","title":"冻结评价回答另一个问题 · 实现代码","unit":"冻结评价回答另一个问题 · 实现代码","unitId":"notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a","chapter":"course","number":1}],"aliases":[{"oldId":"b-49470b7257df9b5e328e3b3d","targetId":"b-49470b7257df9b5e328e3b3d","page":"reading-opening","operation":"move","beforeSHA256":"66fcbfd55e8ee49e9b36e857c1385fb66cc262f5d574a2759c494c801fe22adc","afterSHA256":"66fcbfd55e8ee49e9b36e857c1385fb66cc262f5d574a2759c494c801fe22adc","reason":"Same source body in authored semantic page"},{"oldId":"b-b14e9fb60da4624213c9de7b","targetId":"b-b14e9fb60da4624213c9de7b","page":"reading-opening","operation":"move","beforeSHA256":"60022dcc45f010d088359ab5fd7d3279b36ddd25432e9bff69b3b01c3023b8db","afterSHA256":"60022dcc45f010d088359ab5fd7d3279b36ddd25432e9bff69b3b01c3023b8db","reason":"Same source body in authored semantic page"},{"oldId":"b-61915b8964853fb78f62c77d","targetId":"b-61915b8964853fb78f62c77d","page":"reading-opening","operation":"move","beforeSHA256":"42cba31deccba2a350b9b23bd0ca1dcaa4c4c5cb9d35c773e1fcb89124788ccd","afterSHA256":"42cba31deccba2a350b9b23bd0ca1dcaa4c4c5cb9d35c773e1fcb89124788ccd","reason":"Same source body in authored semantic page"},{"oldId":"b-de4c7121d767f58d532a81a5","targetId":"b-de4c7121d767f58d532a81a5","page":"reading-opening","operation":"move","beforeSHA256":"7d0c53df1bc36d8ae47159a38d1da25d7b95cb99a23f0973fb743325017d6c63","afterSHA256":"7d0c53df1bc36d8ae47159a38d1da25d7b95cb99a23f0973fb743325017d6c63","reason":"Same source body in authored semantic page"},{"oldId":"b-11814d3fc3c1946356a4851f","targetId":"b-11814d3fc3c1946356a4851f","page":"reading-opening","operation":"move","beforeSHA256":"8b9ce5374de71a50bb150ff0a30e2af4bbba003b016c612641bbaafbd706b979","afterSHA256":"8b9ce5374de71a50bb150ff0a30e2af4bbba003b016c612641bbaafbd706b979","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-49470b7257df9b5e328e3b3d","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html#b-49470b7257df9b5e328e3b3d","content":"[LAB 06 · 颜色干预：视图、投影与读出](/course/notebooks/7c4f531113d7473abc174e2e014e1340.html)","sha256":"66fcbfd55e8ee49e9b36e857c1385fb66cc262f5d574a2759c494c801fe22adc","chapter":{"id":"course","title":"冻结评价回答另一个问题 · 实现代码"},"unit":{"id":"notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a","title":"冻结评价回答另一个问题 · 实现代码"},"page":{"id":"reading-opening","title":"冻结评价回答另一个问题 · 实现代码","number":1},"sectionTitle":"冻结评价回答另一个问题 · 实现代码","currentQuestion":"冻结评价回答另一个问题 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-b14e9fb60da4624213c9de7b","prerequisites":[],"relatedBlocks":[]},{"id":"b-b14e9fb60da4624213c9de7b","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html#b-b14e9fb60da4624213c9de7b","content":"# 冻结评价回答另一个问题 · 实现代码","sha256":"60022dcc45f010d088359ab5fd7d3279b36ddd25432e9bff69b3b01c3023b8db","chapter":{"id":"course","title":"冻结评价回答另一个问题 · 实现代码"},"unit":{"id":"notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a","title":"冻结评价回答另一个问题 · 实现代码"},"page":{"id":"reading-opening","title":"冻结评价回答另一个问题 · 实现代码","number":1},"sectionTitle":"冻结评价回答另一个问题 · 实现代码","currentQuestion":"冻结评价回答另一个问题 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-49470b7257df9b5e328e3b3d","nextBlock":"b-61915b8964853fb78f62c77d","prerequisites":[{"id":"b-49470b7257df9b5e328e3b3d","url":"#b-49470b7257df9b5e328e3b3d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-61915b8964853fb78f62c77d","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html#b-61915b8964853fb78f62c77d","content":"主线阅读与优先实验","sha256":"42cba31deccba2a350b9b23bd0ca1dcaa4c4c5cb9d35c773e1fcb89124788ccd","chapter":{"id":"course","title":"冻结评价回答另一个问题 · 实现代码"},"unit":{"id":"notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a","title":"冻结评价回答另一个问题 · 实现代码"},"page":{"id":"reading-opening","title":"冻结评价回答另一个问题 · 实现代码","number":1},"sectionTitle":"冻结评价回答另一个问题 · 实现代码","currentQuestion":"冻结评价回答另一个问题 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-b14e9fb60da4624213c9de7b","nextBlock":"b-de4c7121d767f58d532a81a5","prerequisites":[{"id":"b-b14e9fb60da4624213c9de7b","url":"#b-b14e9fb60da4624213c9de7b","role":"section-title"},{"id":"b-49470b7257df9b5e328e3b3d","url":"#b-49470b7257df9b5e328e3b3d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-de4c7121d767f58d532a81a5","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html#b-de4c7121d767f58d532a81a5","content":"真实RGB对比训练、唯一指定增强变化、h/z形状与颜色读出","sha256":"7d0c53df1bc36d8ae47159a38d1da25d7b95cb99a23f0973fb743325017d6c63","chapter":{"id":"course","title":"冻结评价回答另一个问题 · 实现代码"},"unit":{"id":"notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a","title":"冻结评价回答另一个问题 · 实现代码"},"page":{"id":"reading-opening","title":"冻结评价回答另一个问题 · 实现代码","number":1},"sectionTitle":"冻结评价回答另一个问题 · 实现代码","currentQuestion":"冻结评价回答另一个问题 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-61915b8964853fb78f62c77d","nextBlock":"b-11814d3fc3c1946356a4851f","prerequisites":[{"id":"b-b14e9fb60da4624213c9de7b","url":"#b-b14e9fb60da4624213c9de7b","role":"section-title"},{"id":"b-49470b7257df9b5e328e3b3d","url":"#b-49470b7257df9b5e328e3b3d","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-11814d3fc3c1946356a4851f","type":"pre","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a.html#b-11814d3fc3c1946356a4851f","content":"```\ndef evaluate(model,x,ys,yc,train,test):\n    model.eval()\n    with torch.no_grad(): h,z=model(x)\n    scores={}\n    for space,features in [('h',h),('z',z)]:\n        a=F.normalize(features,dim=1).numpy()\n        for target,y in [('shape',ys),('color',yc)]:\n            knn=KNeighborsClassifier(n_neighbors=5,metric='cosine')\n            knn.fit(a[train],y[train])\n            # Standardization and classifier fit use TRAIN only; test used once for reporting.\n            linear=make_pipeline(StandardScaler(),LogisticRegression(C=1,max_iter=1000))\n            linear.fit(a[train],y[train])\n            scores[f'{space}_{target}_knn']=float(knn.score(a[test],y[test]))\n            scores[f'{space}_{target}_linear']=float(linear.score(a[test],y[test]))\n    with torch.no_grad():\n        # Counterfactual hue swap only: same geometry and pixels, channels exchanged.\n        ah,az=model(x[test][:,[2,1,0]])\n        for space,original,other in [('h',h[test],ah),('z',z[test],az)]:\n            scores[f'{space}_hue_swap_cosine']=float(F.cosine_similarity(original,other).mean())\n    return scores\n\ndef moco_warmup():\n    q=np.array([1.,0.]); keys=np.array([[.8,.2],[0.,1.],[-1.,0.]])\n    keys/=np.linalg.norm(keys,axis=1,keepdims=True)\n    probs={}\n    for T in [.1,.5]:\n        s=keys@q/T; p=np.exp(s-s.max());p/=p.sum();probs[str(T)]=p.tolist()\n    queue=['a','b','c','d']; queue=(queue+['e','f'])[-4:]\n    ema=.9*np.array([0.,2.])+.1*np.array([2.,4.])\n    assert queue==['c','d','e','f']; assert np.allclose(ema,[.2,2.2])\n    return {'temperature_probabilities':probs,'FIFO':queue,'EMA':ema.tolist()}\n\ndef train_moco(initial,x,train,epochs=15):\n    # Small genuine image implementation: query grads, detached EMA keys, FIFO negatives.\n    torch.manual_seed(SEED+2)\n    query=copy.deepcopy(initial); key=copy.deepcopy(initial)\n    for p in key.parameters(): p.requires_grad=False\n    opt=torch.optim.Adam(query.parameters(),lr=.002)\n    queue=torch.empty((0,24)); queue_ids=torch.empty(0,dtype=torch.long)\n    history=[]; trace=[]; m=.95; capacity=240\n    before=next(query.encoder.parameters()).detach().clone()\n    for epoch in range(epochs):\n        losses=[]\n        for ids in torch.as_tensor(train)[torch.randperm(len(train))].split(60):\n            a,b=views(x[ids]),views(x[ids])\n            q=F.normalize(query(a)[1],dim=1)\n            with torch.no_grad():\n                for pk,pq in zip(key.parameters(),query.parameters()):pk.mul_(m).add_(pq,alpha=1-m)\n                k=F.normalize(key(b)[1],dim=1)\n            # Warmup uses detached current keys. Later batches use a real FIFO dictionary.\n            if not len(queue):\n                logits=q@k.T/.2; labels=torch.arange(len(ids))\n                loss=F.cross_entropy(logits,labels)\n            else:\n                positive=(q*k).sum(1,keepdim=True)\n                negative=q@queue.T\n                # A source already in queue is NOT treated as its own negative.\n                negative=negative.masked_fill(ids[:,None]==queue_ids[None,:],-1e9)\n                logits=torch.cat([positive,negative],dim=1)/.2\n                loss=F.cross_entropy(logits,torch.zeros(len(ids),dtype=torch.long))\n            opt.zero_grad();loss.backward();opt.step()\n            with torch.no_grad():\n                queue=torch.cat([queue,k])[-capacity:];queue_ids=torch.cat([queue_ids,ids])[-capacity:]\n            losses.append(float(loss.detach()))\n            if len(trace)<5:trace.append({'step':len(trace),'queue_size':len(queue),'first_source_id':int(queue_ids[0]),'last_source_id':int(queue_ids[-1])})\n        history.append(float(np.mean(losses)))\n    assert all(p.grad is None for p in key.parameters())\n    return query,key,queue,queue_ids,history,trace,float((next(query.encoder.parameters()).detach()-before).norm())\n\ndef plot_results(out,x,initial,models,results,histories):\n    fig,ax=plt.subplots(3,6,figsize=(12,6))\n    ids=[0,100,140,280,420,700]\n    torch.manual_seed(SEED+6)\n    for j,i in enumerate(ids):\n        ax[0,j].imshow(x[i].permute(1,2,0));ax[0,j].set_title(f'source {i}')\n        paired_state=torch.get_rng_state()\n        for row,erase in [(1,False),(2,True)]:\n            torch.set_rng_state(paired_state)\n            ax[row,j].imshow(views(x[i:i+1],erase)[0].permute(1,2,0))\n    for row,label in enumerate(['source','view: hue kept','view: hue can swap']):ax[row,0].set_ylabel(label)\n    for a in ax.flat:a.set_xticks([]);a.set_yticks([])\n    fig.tight_layout();fig.savefig(out/'views.png',dpi=140);plt.close(fig)\n    fig,axes=plt.subplots(1,3,figsize=(14,4))\n    for name,hist in histories.items():axes[0].plot(np.arange(1,len(hist)+1),hist,label=name)\n    axes[0].set(xlabel='epoch',ylabel='InfoNCE (batch mean)',title='Training objective');axes[0].legend(fontsize=8)\n    names=list(results)\n    for ax,task in zip(axes[1:],['shape','color']):\n        pos=np.arange(len(names)); width=.34\n        for off,space in [(-width/2,'h'),(width/2,'z')]:\n            ax.bar(pos+off,[results[n][f'{space}_{task}_knn'] for n in names],width,label=space)\n        ax.set_xticks(pos,names,rotation=20);ax.set_ylim(0,1.05);ax.set_title(f'Frozen cosine 5-NN: {task}')\n        ax.set_ylabel('held-out accuracy');ax.legend()\n    fig.tight_layout();fig.savefig(out/'training-and-probes.png',dpi=140);plt.close(fig)\n\ndef run_contrastive(out_dir='lab06-output',mode='color'):\n    started=time.time();out=Path(out_dir);out.mkdir(parents=True,exist_ok=True)\n    torch.manual_seed(SEED)\n    x,ys,yc,train,test=make_data()\n    np.savez_compressed(out/'inputs.npz',images=x.numpy(),shape=ys,color=yc,train_ids=train,test_ids=test)\n    initial=ContrastiveModel(); torch.save(initial.state_dict(),out/'random-init.pt')\n    results={'random':evaluate(initial,x,ys,yc,train,test)};histories={};models={};deltas={}\n    if mode=='color':\n        for name,erase in [('keep-hue',False),('swap-hue',True)]:\n            model,hist,delta=train_simclr(initial,x,train,erase)\n            results[name]=evaluate(model,x,ys,yc,train,test);histories[name]=hist;models[name]=model;deltas[name]=delta\n            torch.save(model.state_dict(),out/f'{name}.pt')\n    else:\n        model,hist,delta=train_simclr(initial,x,train,False)\n        results['simclr']=evaluate(model,x,ys,yc,train,test);histories['simclr']=hist;models['simclr']=model;deltas['simclr']=delta\n        torch.save(model.state_dict(),out/'simclr.pt')\n        query,key,queue,queue_ids,hist,trace,delta=train_moco(initial,x,train)\n        results['moco-toy']=evaluate(query,x,ys,yc,train,test);histories['moco-toy']=hist;models['moco-toy']=query;deltas['moco-toy']=delta\n        torch.save({'query':query.state_dict(),'key':key.state_dict(),'queue':queue,'queue_ids':queue_ids},out/'moco-toy.pt')\n        (out/'queue-trace.json').write_text(json.dumps(trace,indent=2))\n    config={'seed':SEED,'mode':mode,'image_shape':[3,16,16],'train_n':len(train),'test_n':len(test),'batch_size':60,'epochs_simclr':40,'epochs_moco':15,'tau':.2,'adam_lr':.002,'h_dim':48,'z_dim':24,'labels_used_only_for_probes':True,'knn':{'k':5,'metric':'cosine'},'linear_probe':{'regularization_C':1,'standardizer_fit':'train-only'},'test_tuning':'none; fixed protocol before first run'}\n    (out/'config.json').write_text(json.dumps(config,indent=2))\n    plot_results(out,x,initial,models,results,histories)\n    payload={'code_sha256':globals().get('SOURCE_SHA256') or hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),'kind':'local CPU full training + frozen held-out probes','utc':time.strftime('%Y-%m-%dT%H:%M:%SZ',time.gmtime()),'seconds':time.time()-started,'torch':torch.__version__,'numpy':np.__version__,'python':platform.python_version(),'scores':results,'loss_history':histories,'encoder_weight_l2_change':deltas,'warmup':moco_warmup(),'inputs_sha256':hashlib.sha256((out/'inputs.npz').read_bytes()).hexdigest()}\n    (out/'results.json').write_text(json.dumps(payload,indent=2))\n    print(json.dumps({'scores':results,'encoder_weight_l2_change':deltas,'seconds':payload['seconds']},indent=2))\n    return payload\n```","sha256":"8b9ce5374de71a50bb150ff0a30e2af4bbba003b016c612641bbaafbd706b979","chapter":{"id":"course","title":"冻结评价回答另一个问题 · 实现代码"},"unit":{"id":"notebook-blocks/cdcacb55364648fe84ae33c18ee97f1a","title":"冻结评价回答另一个问题 · 实现代码"},"page":{"id":"reading-opening","title":"冻结评价回答另一个问题 · 实现代码","number":1},"sectionTitle":"冻结评价回答另一个问题 · 实现代码","currentQuestion":"冻结评价回答另一个问题 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-de4c7121d767f58d532a81a5","nextBlock":null,"prerequisites":[{"id":"b-b14e9fb60da4624213c9de7b","url":"#b-b14e9fb60da4624213c9de7b","role":"section-title"},{"id":"b-49470b7257df9b5e328e3b3d","url":"#b-49470b7257df9b5e328e3b3d","role":"problem-context"}],"relatedBlocks":[]}]}