64 Bytes per Document: Shrinking EmbeddingGemma 2 Vectors for Japanese Search
· 16 min read
A million Japanese passages take 3 GB of RAM as EmbeddingGemma 2 vectors. I got them down to 64 MB at 98% of the search quality. The trick that matters most: compress how you store vectors, never how you score them.