{"version":2,"relationshipURLBase":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html","relationshipResolution":"Resolve relative relationship URLs against relationshipURLBase; complete records and absolute URLs are in blocks.","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html","markdown":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.md","pages":[{"id":"reading-opening","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","unit":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","unitId":"notebook-blocks/c21718fad7a840a380c87175f7067cf5","chapter":"course","number":1}],"aliases":[{"oldId":"b-d930a9be3f9c4a3ea41a57c6","targetId":"b-d930a9be3f9c4a3ea41a57c6","page":"reading-opening","operation":"move","beforeSHA256":"db08960319942d9119c9cad2cf7d0b78a68c914f8e19ab3bd375531d27c96204","afterSHA256":"db08960319942d9119c9cad2cf7d0b78a68c914f8e19ab3bd375531d27c96204","reason":"Same source body in authored semantic page"},{"oldId":"b-42b021be766ea5ec8a980031","targetId":"b-42b021be766ea5ec8a980031","page":"reading-opening","operation":"move","beforeSHA256":"07188122906337190e1fbd57d96cd45f3ef5e2c4da0ed0475ef2758b81774d88","afterSHA256":"07188122906337190e1fbd57d96cd45f3ef5e2c4da0ed0475ef2758b81774d88","reason":"Same source body in authored semantic page"},{"oldId":"b-7fae0bb5405c80ba808266e9","targetId":"b-7fae0bb5405c80ba808266e9","page":"reading-opening","operation":"move","beforeSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","afterSHA256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","reason":"Same source body in authored semantic page"},{"oldId":"b-375007f5125d2400ee05584b","targetId":"b-375007f5125d2400ee05584b","page":"reading-opening","operation":"move","beforeSHA256":"6a611aef342a7535362f318ee0c33b63170ce4ab853fa44a05f76a04873fd320","afterSHA256":"6a611aef342a7535362f318ee0c33b63170ce4ab853fa44a05f76a04873fd320","reason":"Same source body in authored semantic page"},{"oldId":"b-e0157173e9bf999e3f86141c","targetId":"b-e0157173e9bf999e3f86141c","page":"reading-opening","operation":"move","beforeSHA256":"a8795f74087fc20755c646a3e622120516eb1f0cf95407daf7336d82452e3833","afterSHA256":"a8795f74087fc20755c646a3e622120516eb1f0cf95407daf7336d82452e3833","reason":"Same source body in authored semantic page"}],"blocks":[{"id":"b-d930a9be3f9c4a3ea41a57c6","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html#b-d930a9be3f9c4a3ea41a57c6","content":"[LAB 13 · 看图生成一句话：训练、解码与消融](/course/notebooks/3da47f3ee3554ee0a36e0bb6a2fdc61f.html)","sha256":"db08960319942d9119c9cad2cf7d0b78a68c914f8e19ab3bd375531d27c96204","chapter":{"id":"course","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"unit":{"id":"notebook-blocks/c21718fad7a840a380c87175f7067cf5","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"page":{"id":"reading-opening","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","number":1},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":null,"nextBlock":"b-42b021be766ea5ec8a980031","prerequisites":[],"relatedBlocks":[]},{"id":"b-42b021be766ea5ec8a980031","type":"h1","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html#b-42b021be766ea5ec8a980031","content":"# 4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","sha256":"07188122906337190e1fbd57d96cd45f3ef5e2c4da0ed0475ef2758b81774d88","chapter":{"id":"course","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"unit":{"id":"notebook-blocks/c21718fad7a840a380c87175f7067cf5","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"page":{"id":"reading-opening","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","number":1},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-d930a9be3f9c4a3ea41a57c6","nextBlock":"b-7fae0bb5405c80ba808266e9","prerequisites":[{"id":"b-d930a9be3f9c4a3ea41a57c6","url":"#b-d930a9be3f9c4a3ea41a57c6","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-7fae0bb5405c80ba808266e9","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html#b-7fae0bb5405c80ba808266e9","content":"按问题继续","sha256":"e91f24e4e4ca40e2099a0d81d30db6568f4bf2074b499eeee6e3959617bd97ac","chapter":{"id":"course","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"unit":{"id":"notebook-blocks/c21718fad7a840a380c87175f7067cf5","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"page":{"id":"reading-opening","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","number":1},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-42b021be766ea5ec8a980031","nextBlock":"b-375007f5125d2400ee05584b","prerequisites":[{"id":"b-42b021be766ea5ec8a980031","url":"#b-42b021be766ea5ec8a980031","role":"section-title"},{"id":"b-d930a9be3f9c4a3ea41a57c6","url":"#b-d930a9be3f9c4a3ea41a57c6","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-375007f5125d2400ee05584b","type":"p","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html#b-375007f5125d2400ee05584b","content":"真实CNN–GRU条件训练与greedy/beam生成、遮图/错图消融","sha256":"6a611aef342a7535362f318ee0c33b63170ce4ab853fa44a05f76a04873fd320","chapter":{"id":"course","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"unit":{"id":"notebook-blocks/c21718fad7a840a380c87175f7067cf5","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"page":{"id":"reading-opening","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","number":1},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-7fae0bb5405c80ba808266e9","nextBlock":"b-e0157173e9bf999e3f86141c","prerequisites":[{"id":"b-42b021be766ea5ec8a980031","url":"#b-42b021be766ea5ec8a980031","role":"section-title"},{"id":"b-d930a9be3f9c4a3ea41a57c6","url":"#b-d930a9be3f9c4a3ea41a57c6","role":"problem-context"}],"relatedBlocks":[]},{"id":"b-e0157173e9bf999e3f86141c","type":"pre","scope":"document","url":"https://codingai-lec03.pages.dev/course/notebook-blocks/c21718fad7a840a380c87175f7067cf5.html#b-e0157173e9bf999e3f86141c","content":"```\n@torch.no_grad()\ndef greedy(pixels, prefix=(BOS,), max_new=8):\n    model.eval(); seq=torch.tensor(prefix).repeat(len(pixels),1); finished=torch.zeros(len(pixels),dtype=torch.bool)\n    for _ in range(max_new):\n        nxt=model(pixels,seq)[:,-1].argmax(-1)\n        nxt=torch.where(finished,torch.tensor(EOS),nxt)\n        seq=torch.cat((seq,nxt[:,None]),1); finished|=nxt.eq(EOS)\n        if bool(finished.all()): break\n    return seq.tolist()\n\n@torch.no_grad()\ndef beam_one(pixel, width=3, max_new=8):\n    beams=[([BOS],0.0,False)]\n    for _ in range(max_new):\n        candidates=[]\n        for seq,score,done in beams:\n            if done: candidates.append((seq,score,True)); continue\n            lp=F.log_softmax(model(pixel[None],torch.tensor([seq]))[0,-1],dim=-1)\n            values,indices=lp.topk(width)\n            for value,idx in zip(values.tolist(),indices.tolist()):\n                candidates.append((seq+[idx],score+value,idx==EOS))\n        beams=sorted(candidates,key=lambda b:b[1],reverse=True)[:width]\n        if all(b[2] for b in beams): break\n    # Ended hypotheses only if at least one ended; otherwise max-length fallback.\n    ended=[b for b in beams if b[2]]\n    return max(ended or beams,key=lambda b:b[1])\n\ndef words(ids):\n    out=[]\n    for idx in ids:\n        if idx==EOS: break\n        if idx not in [BOS,PAD]: out.append(VOCAB[idx])\n    return ' '.join(out)\n\npred=[words(s) for s in greedy(Xtest)]\nbeam_pred=[words(beam_one(x)[0]) for x in Xtest]\nzero_pred=[words(s) for s in greedy(torch.zeros_like(Xtest))]\n# A deterministic derangement shifts test images; target captions stay fixed.\nshuffle_index=torch.roll(torch.arange(len(Xtest)),1)\nshuffle_pred=[words(s) for s in greedy(Xtest[shuffle_index])]\nexact=lambda p: float(np.mean([a==b for a,b in zip(p,test_text)]))\nmetrics={'heldout_images':len(Xtest),'greedy_exact_match':exact(pred),'beam3_exact_match':exact(beam_pred),'zero_pixels_exact_match':exact(zero_pred),'shuffled_pixels_exact_match':exact(shuffle_pred),'final_train_token_ce':curve[-1]['train_token_ce'],'final_heldout_teacher_forced_ce':curve[-1]['heldout_teacher_forced_ce']}\nprint(json.dumps(metrics,indent=2))\n# Hold the language prefix fixed; change only the input image.\nrng=np.random.default_rng(999)\ncontrol_specs=[('red','square','left'),('blue','circle','right'),('green','square','center')]\ncontrol_pixels=torch.tensor(np.stack([render_object(*spec,rng) for spec in control_specs])).permute(0,3,1,2)\ncontrol_pred=[words(s) for s in greedy(control_pixels,prefix=(BOS,W['a']))]\ncontrols=[{'condition':list(spec),'fixed_prefix':'<bos> a','generated':p} for spec,p in zip(control_specs,control_pred)]\nprint('same prefix, changed image:',json.dumps(controls,indent=2))\n```","sha256":"a8795f74087fc20755c646a3e622120516eb1f0cf95407daf7336d82452e3833","chapter":{"id":"course","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"unit":{"id":"notebook-blocks/c21718fad7a840a380c87175f7067cf5","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码"},"page":{"id":"reading-opening","title":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","number":1},"sectionTitle":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","currentQuestion":"4. 推理只接自己的输出，再检查视觉依赖 · 实现代码","sourceSlides":[],"knowledgeBlock":null,"formulas":[],"previousBlock":"b-375007f5125d2400ee05584b","nextBlock":null,"prerequisites":[{"id":"b-42b021be766ea5ec8a980031","url":"#b-42b021be766ea5ec8a980031","role":"section-title"},{"id":"b-d930a9be3f9c4a3ea41a57c6","url":"#b-d930a9be3f9c4a3ea41a57c6","role":"problem-context"}],"relatedBlocks":[]}]}