TIES-Merging under the hood: how sign conflicts and parameter interference are resolved
TIES-Merging improves over naive averaging by trimming low-magnitude delta weights, electing a dominant sign across models, and then merging only sign-aligned parameters — this directly targets both redundancy and sign interference — but it still assumes the component models remain sufficiently compatible in weight space.