34 points | by ifz 1 hour ago
4 comments
are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.
are you worried later-layer attention gets drowned out by earlier layers just because there are more of them contributing to the sum?
To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.