pruning-head-path-patching-no-weight-update

IN premise — summaries/2026/08/24/xu-2024-knowledge-conflicts-survey-s5-challenges-and-future-directions.md

Created 2026-08-25T02:59:03+00:00

Pruning Head via Path Patching (Jin et al., 2024b) identifies attention heads with opposite effects in later layers and prunes conflicting heads without updating model weights.

Summary

This describes a technique for trimming a transformer model by finding attention heads in later layers that pull the output in opposite directions and removing the ones causing the most conflict, all in a single step without retraining any weights. It matters because you get a leaner, more coherent model from one surgical intervention rather than a costly fine-tuning cycle.