DynamicLRP: The Future of Neural Network Attribution
DynamicLRP sets a new standard for neural network attribution, making Layer-wise Relevance Propagation accessible across architectures without model changes.
DynamicLRP is shaking up neural network attribution by breaking away from the constraints of architecture-specific Layer-wise Relevance Propagation (LRP). Existing implementations bind users to module-level operations, demanding cumbersome model alterations that don't keep pace as architectures evolve. Enter DynamicLRP, a framework that operates at the tensor operation level, liberating LRP from these constraints while maintaining its theoretical rigor.
Agnostic and Extensible
What makes DynamicLRP a breakthrough? It's model-agnostic, meaning it doesn’t need any architecture-specific propagation rules. By decomposing attribution to individual operations within computation graphs, it achieves true agnosticity. This is accomplished through a novel mechanism known as the Promise System, which defers activation resolution efficiently.
The paper's key contribution is its ability to sidestep the traditional backpropagation machinery. This allows DynamicLRP to run alongside gradient backpropagation without any need for model modifications. It's underpinned by computation graphs, making it theoretically extensible to other deep learning libraries that support auto-differentiation.
Performance and Coverage
Performance-wise, DynamicLRP delivers. It matches or even exceeds specialized implementations with an attribution accuracy of 93.70% and 95.06% for explaining RoBERTa-large and Flan-T5-large on SQuADv2, respectively. This is no small feat and highlights its efficiency across models with 100M to 1B parameters. DynamicLRP achieves 99.92% node coverage over 31,465 computation graph nodes from 15 diverse architectures, without any model-specific code. That's a significant achievement.
Why It Matters
Why should we care? Because as deep learning architectures diversify, maintaining flexible and sustainable attribution methods is essential. DynamicLRP's operation-level decomposition and Promise System mean it doesn’t just work today but paves the road for tomorrow’s architectures without constantly needing to reinvent the wheel.
DynamicLRP challenges the status quo. Will it redefine what's possible in neural network attribution? Its ability to provide a sustainable, extensible foundation for LRP suggests that it just might. Code and data are available at https://github.com/keeinlev/dynamicLRP, inviting further exploration and innovation.
Get AI news in your inbox
Daily digest of what matters in AI.
Key Terms Explained
The algorithm that makes neural network training possible.
A subset of machine learning that uses neural networks with many layers (hence 'deep') to learn complex patterns from large amounts of data.
A computing system loosely inspired by biological brains, consisting of interconnected nodes (neurons) organized in layers.