64 Bytes per Document: Shrinking EmbeddingGemma 2 Vectors for Japanese Search

A million Japanese passages, embedded with EmbeddingGemma 2 at full size, take 3 GB of RAM. I got the same million down to 64 MB and kept 98% of the search quality.

On the way I found that the most common way to store vectors as int8 quietly loses up to 8 points of quality. The int8 numbers themselves are fine. The loss comes from how they are compared.

Why I ran this

EmbeddingGemma 2 came out on October 6. Its model card says you can cut its 768-number vectors down to 256 numbers and stay "close to lossless". That claim is an average over many languages, and it is measured before any compression. Real vector databases also compress each number, to one byte or to a single bit. Nobody had published what the combination does to Japanese search, so I measured every combination on one Japanese benchmark.

The short version: compress how you store vectors, not how you score them.

There are three ways to make a vector smaller, and this post has one section for each:

1. Keep fewer numbers (Matryoshka truncation). Almost free down to 256 of 768; broken at 128.
2. Store each number in one byte (int8). Lossless, if you decode the bytes before scoring.
3. Store each number as one bit (binary). Weak on its own, but with a second step it gives the 64-byte result above.

After that, an interactive widget combines all three, and the post ends with the setup I would ship. If you pay for a vector database by the gigabyte, or want search to run on the machine you already have, these three choices decide the bill.

How I measured

The benchmark is JQaRA: 1,667 Japanese quiz questions and 144,372 Wikipedia passages. For each question, a few passages are labelled as containing the answer.

For every question I searched all 144,372 passages and kept the top 100, the way a production index would. (JQaRA's official protocol only re-orders 100 pre-selected passages per question. I ran it too; it leads to the same conclusions, and the numbers are in the results tables.)

The quality metric is nDCG@10. It goes from 0 to 1 and is higher when a correct passage appears near the top of the first 10 results. To keep things readable I mostly report it as a percentage of the full-size model: 100% means 768 numbers in float32, nDCG@10 = 0.567.

Three rules keep the comparison fair:

• Encode once. Every passage and question was embedded once at full size. Every smaller version is cut and compressed from that same output.
• Exact search only. No approximate index (no HNSW, no IVF), so a difference between two results comes from the vector and nothing else.
• Same labels everywhere. A passage counts as correct only for the question it was shipped with, so absolute scores are a bit pessimistic. Every configuration is judged against the same labels, so the comparisons hold.

As a sanity check, EmbeddingGemma 2 scores 0.583 in JQaRA's official protocol: above multilingual-e5-large (0.554), below the Japanese-specific ruri-large (0.629). BM25, the classic keyword search, reaches 84% of the full-size model in my setup. It shows up as a reference line below.

1. Keep fewer numbers: free down to 256

EmbeddingGemma 2 is trained with Matryoshka representation learning. The model learns to pack the most useful information into the first numbers of the vector, so the first 256 numbers already work as an embedding on their own. A progressive JPEG works the same way: the first bytes give you a blurry but complete picture. To shrink a vector, you keep its first k numbers and rescale them to length 1.

Here is what that costs on JQaRA, still in float32:

The model card's claim holds for Japanese down to 256. At 128 the curve falls off: 82% is about where BM25 sits (84%). In JQaRA's official protocol, 128-number EmbeddingGemma 2 and BM25 score the same, 0.510 and 0.511. At that size you are running a neural network to do a keyword index's job.

So: cut to 512 for free, to 256 for 5%, and stop there. The next two sections shrink each of those numbers.

2. One byte per number: lossless, if you decode first

How int8 works

A float32 number takes 4 bytes. int8 stores it in 1 byte, as one of 256 levels from −128 to 127. The usual recipe, for example quantize_embeddings in sentence-transformers, works one dimension at a time:

1. Look at all documents and find the smallest and largest value of that dimension.
2. Split that range into 256 equal steps. The smallest value becomes −128, the largest 127.
3. Store each document's value as the number of its step.

A concrete dimension: say its values across the corpus run from −0.02 to 0.10. Then −0.02 is stored as −128, 0.10 as 127, and the middle of the range, 0.04, as about 0. A value of exactly 0 is stored as about −85.

Keep that last detail in mind. Code 0 does not mean 0. It means "the middle of this dimension's range", and the middle is usually not 0.

Two ways to score the same bytes

Once documents are stored as int8, there are two ways to compare a query with them:

• A. Code against code. Convert the query to int8 with the same ranges, and take the dot product of the two int8 vectors. It is fast and fits int8 CPU instructions. Most examples do this.
• B. Decode, then score. Turn each document's codes back into approximate floats with the stored ranges, and take the dot product with the float32 query.

Same bytes on disk, different results:

Decoding matches float32 to the third decimal at every size. Scoring code against code loses 3.5 to 7.6 points.

Why code against code reorders results

Go back to the dimension from the example above. Its range is −0.02 to 0.10, the middle is 0.04, and code 0 stands for 0.04. A code does not store the value itself. It stores how many steps the value sits above or below that middle:

code ≈ (value − middle) / step

Now look at what scoring A multiplies. It does not multiply query values by document values. It multiplies their distances from the middle. Over the whole vector, with m the list of per-dimension middles, scoring A ranks documents by (q − m) · (d − m) instead of the true q · d. (Dividing by the step also re-weights dimensions; that turns out to matter less, see below.)

Moving the reference point from zero to m sounds harmless, but it can change which document wins. Here is a two-dimensional example. Start with the true scores:

Document 1 is identical to the query, so it should win. Now say the middles of the two dimensions are m = (0.3, 0.7). Subtract m from every vector, which is what the codes do, and score again:

Document 2 now wins. Expanding the shifted dot product shows the reason:

(q − m) · (d − m)  =  q · d  −  m · d  +  (a term that is the same for every document)

Every document pays a penalty m · d, whatever the query is. The more a document points toward m, the typical direction of the corpus, the more it loses. Document 1 points closer to m (penalty 0.74 against 0.66), so it drops below document 2.

Real embeddings are far from zero-centered. All JQaRA passages share a common direction; in an average dimension, the middle of the range sits about 0.6 standard deviations away from zero. So the penalty is real, and it pushes typical-looking passages down.

Is the shift really the culprit? Code-against-code scoring changes three things at once: it subtracts m, it divides each dimension by its own step size, and it rounds to whole numbers. I applied these to the float vectors one at a time and searched again, at 768 numbers:

The shift alone costs 2.6 of the 3.5 points. Dividing by the step costs the rest. Rounding costs nothing: the last two rows are the same. The grey line in the chart above is the "subtract m" row, at every size.

The fix

Keep the codes. Store two floats per dimension for the whole index (the smallest and largest value), and score a float query against decoded documents:

codes = quantize_embeddings(docs, precision="int8", calibration_embeddings=docs)

lo, hi = docs.min(axis=0), docs.max(axis=0)
docs_hat = (codes.astype(np.float32) + 128) * (hi - lo) / 255 + lo

scores = query_fp32 @ docs_hat.T   # matches float32 quality

You do not need to build docs_hat in memory. Decoding is linear, so you can multiply the float query by the step size once and take its dot product with the raw codes; what is left over is the same for every document and does not change the ranking.

Does your vector database already do this? That depends on how it quantizes, and I did not test any. If anything in your pipeline takes the dot product of two int8 vectors, check it. On this data, the fix was worth 3 to 8 points.

So: int8 gives you 4× smaller vectors for free, if the query stays in float. Combined with section 1, 256 numbers in int8 is 256 bytes at 95%. To go further, we need bits.

3. One bit per number: weak alone, strong with a second step

Binary quantization keeps only the sign of each number: 1 if it is positive, 0 if not. A 512-number vector becomes 512 bits, which is 64 bytes. Two binary vectors are compared by the Hamming distance, the count of bits where they differ. A CPU computes it with an XOR and a bit count, so it is very fast.

On its own, binary search is not good enough below full size:

Rescoring: search with bits, rank with bytes

The fix is to split one search into two steps:

1. Find candidates with bits. Binary codes of all documents live in RAM. The query is converted to bits the same way, compared with every document by Hamming distance, and the 100 closest are kept. (I used an exact FAISS IndexBinaryFlat search.)
2. Re-rank the 100 with bytes. Read those 100 documents' int8 codes from disk, decode them as in section 2, and score them against the float32 query. Sort by that score.

RAM holds only the bits. The int8 copy is read 100 vectors per query, so it can live on an SSD.

At 512 numbers this gives 64 bytes in RAM at 98% of full quality. That is the 64 MB per million documents from the first line of this post.

Three limits to know:

• It cannot find what step 1 missed. Re-ranking only re-orders the 100 candidates. At 512 numbers, the binary top 100 contains 75% of the correct passages, against 89% for float32. If your RAG pipeline passes the top 5 to an LLM, the top-10 quality above is what matters. If something downstream needs all 100, fetch more binary candidates first. I only tested 100.
• It breaks down at 256 and below. There the right passage is often not among the binary top 100 at all, and rescoring reaches only 90% (256) and 66% (128).
• float32 on disk does not help. Re-ranking with float32 instead of int8 scores 0.5557 against 0.5555. Keep int8 on disk; it is 4× smaller.

So: bits are a cheap first filter, and decoded bytes make the final ranking.

All three together

Here are all three knobs on one real passage embedding. Every number in the widget is measured. The vertical tick on the bar is BM25.

And every configuration on one chart, by memory against quality:

Before you copy this setup

Three findings that do not change the index size, but change what you should deploy.

EmbeddingGemma 1 is better at Japanese. I ran the same grid for the previous model and expected it to lose. It wins at every size and precision: 0.592 against 0.567 at full size.

The model card agrees if you read it closely: EmbeddingGemma 2's multilingual average barely moved (61.36 against 61.15). Its gains are in code search (78.7 against 68.8) and in images, video and audio.

Adding BM25 does not help here. Mixing BM25 and dense rankings (reciprocal rank fusion) gives 0.5674 against 0.5668 for the dense model alone. BM25 is 10 points behind on this dataset, so it has nothing to add. A corpus full of product codes or rare names could behave differently.

The query encoder can run on a CPU. Every query is embedded at request time. I exported the text encoder to ONNX, made an int8 copy with ONNX Runtime, and timed real JQaRA questions on an M1 Pro with 8 threads:

For one query at a time, the int8 encoder is twice as fast and 4× smaller. It costs about 1.2 points of nDCG@10 (0.567 to 0.555). These are laptop numbers measured with other software running; trust the ratios more than the milliseconds.

What I would ship

We started at 3 GB for a million passages and ended at 64 MB, because each step compressed storage and left the similarity alone: truncation keeps the model's own geometry, int8 is decoded before scoring, and bits only pick candidates for a float-quality re-rank.

1. Default: 256 or 512 numbers, int8 codes, decoded for scoring. 6 to 12× smaller than full size at 95 to 99% of the quality, with any exact or IVF index.
2. When RAM is the limit: 512-number binary in RAM, int8 on disk, re-rank the top candidates. 64 MB per million documents at 98%. Fetch more than 100 candidates if recall matters downstream.
3. For Japanese-only text search, use EmbeddingGemma 1. Pick EmbeddingGemma 2 when the same index must hold code or images. Avoid 128 numbers for Japanese.

One dataset is not the world. JQaRA is quiz questions over Wikipedia, and legal text or support tickets may react differently to truncation. The int8 penalty, though, only needs embeddings that are not centered at zero, and that is common.

So here is one thing to do this week: find the line in your search pipeline where int8 vectors get compared. If both sides are int8 codes, decode the documents and keep the query in float. It needs no extra storage and gave me back up to 8 points.

The code, cached results and every chart are in HiGal/mrl-quant-ja. make encode builds the embeddings (about 1.5 hours per model on Apple silicon), and make reproduce regenerates every number in this post in about two minutes. If you run it with another model or on your own vector database, I would like to know whether the penalty shows up there too.