Gemma Scope 2 Architecture: Decapsulating LLM Behavior via Sparse Autoencoders
Gemma Scope 2 Architecture: Decapsulating LLM Behavior via Sparse Autoencoders Gemma Scope 2 Architecture: Decapsulating LLM Behavior via Sparse Autoencoders Executive Analysis: The release of Gemma Scope 2 marks a pivotal moment in the transition from opaque stochastic parrots to decipherable cognitive engines. By leveraging JumpReLU Sparse Autoencoders (SAEs) on the Gemma 2 27B, 9B,