← Back to Vault

Nested Active Parameters

Cameron Rohn · Category: frameworks_and_exercises

Gemma 3N uses a nested architecture with 2 billion active parameters out of 4 billion total to drastically cut computational requirements while retaining performance.