Unpacking the Black Box: Cross-Layer Transcoders' Role in Vision Transformers

Vision Transformers (ViTs) have revolutionized computer vision, but understanding their internal workings remains a challenge. While Sparse Autoencoders (SAEs) have been used to extract interpretable features, they fall short in capturing the full computational picture across multiple layers. Enter Cross-Layer Transcoders (CLTs), offering a fresh perspective on making these models more transparent and trustworthy.

The Rise of Cross-Layer Transcoders

CLTs serve as sparse, depth-aware proxy models for MLP blocks within ViTs. Crucially, they don't just reconstruct activations of individual layers. They also account for how each layer contributes to the final output. This approach provides what SAEs can't: a linear decomposition that turns ViT’s opaque embeddings into a clear, additive structure. The paper, published in Japanese, reveals that CLTs can transform the final representation of ViTs, allowing for process-level interpretability and faithful attribution.

Benchmarking CLTs

Researchers trained CLTs on CLIP ViT-B/32 and ViT-B/16 models using datasets like CIFAR-100, COCO, and ImageNet-100. The benchmark results speak for themselves. CLTs not only achieve high fidelity in reconstructing post-MLP activations, but they also maintain, and occasionally enhance, the zero-shot classification accuracy of CLIP. Compare these numbers side by side: CLTs redefine how we can interpret the inner workings of ViTs.

Implications for the Vision Domain

Western coverage has largely overlooked this but understanding cross-layer contributions is essential. The data shows that a concentrated set of dominant layer-wise terms are responsible for the majority of the final representation. Remove these, and performance degrades. Retain them, and it largely remains intact. This raises an intriguing question: could focusing only on these key layers speed up computational efficiency in future models?

CLTs are more than just an alternative. they're a necessary evolution for interpretability in vision models. As researchers and developers seek trustworthy AI, the need for transparent models becomes increasingly urgent. CLTs offer a promising direction.