EmbeddingGemma 2: The Whole-System Cost of Local Multimodal Search

I would choose a local search engine by the collection it can usefully search, rather than the smallest memory number attached to its model. EmbeddingGemma 2 makes that distinction consequential: its most compact vectors save sixfold in raw vector storage, but Google’s own results show a much larger penalty for multimodal retrieval than for text.

Reporting on the October 6 launch, Ana Maria Constantin at The Next Web emphasized local control using existing hardware. That is a compelling reason to consider the release. It also raises the question that matters after the model downloads: what else must an application keep and run locally for that control to produce useful results?

Google DeepMind released EmbeddingGemma 2 under Apache 2.0, with text, image, video and audio inputs mapped into a shared 768-dimensional space. An embedding represents an input as a vector for tasks such as similarity search; it does not write an answer. The 740-million-parameter model has a 270-million-parameter text component and optional vision and audio components of 170 million and 300 million parameters.

Google reports roughly 191 megabytes of active RAM for quantized text-only weights and 567 megabytes for the full multimodal model’s weights on a Pixel 11 Pro. Those figures describe weights on that device, not the complete application. As a collection grows, its vectors, metadata and source media add a separate resource demand. The model can fit while the search product still needs difficult compromises.

Sixfold savings come with a different search tradeoff

At the core of EmbeddingGemma 2 is Matryoshka Representation Learning, a technique formalized by Kusupati and colleagues in 2022. Matryoshka models learn useful representations in nested subsets of a vector. A single 768-dimensional output vector contains valid lower-dimensional representations in its leading 512, 256, or 128 elements. Downstream systems can shorten these vectors by simply truncating the trailing dimensions and renormalizing, without running the encoder again if the longer vectors have been retained. Discarded dimensions cannot later be recovered from a saved short vector alone. Queries and indexed items must use matching dimensions.

This design presents an obvious opportunity to trim index memory. A 128-dimensional vector requires one-sixth the raw memory of a full 768-dimensional vector. The same saving applies to the raw vector arrays, rather than every part of an index. However, Google’s own full-precision evaluation disclosures demonstrate that this compression does not treat all modalities equally.

The model card publishes evaluation figures across three distinct standardized benchmarks: multilingual MTEB v2, MMEB v2 covering multimodal tasks, and MSEB audio retrieval. These benchmarks measure distinct evaluation metrics and cannot be directly compared against one another, but tracking how each score declines across dimension thresholds reveals where the larger losses appear.

Google-reported full-precision checkpoint results. Each benchmark has its own metric; compare changes within a column, rather than values across columns. The storage ratio compares full-length and retained vector size at the same component precision.

On multilingual MTEB v2, cutting the representation from 768 to 128 dimensions changes the mean score from 61.36 to 57.89. Google identifies 128 dimensions as best suited to text-only workloads. That guidance is useful, although the table does not establish why text behaves differently or how much loss a particular application can tolerate.

The multimodal scores fall further. On MMEB v2 overall, the score falls by 13.36 points when moving from 768 dimensions to 128 dimensions, dropping from 59.01 to 45.65. On audio tasks via MSEB, the retrieval score drops from 69.54 to 56.71, a loss of 12.83 points.

The decline is not linear. Moving from 768 down to 256 dimensions cuts the raw vector footprint by two-thirds while MMEB v2 drops by only 2.77 points (from 59.01 to 56.24) and MSEB drops by 2.78 points. The larger multimodal decline comes in that final step from 256 down to 128 dimensions, where MMEB sheds 10.59 points in a single cut.

For a product chosen specifically to search photos or recordings with natural-language queries, that final saving deserves scrutiny. I would start by comparing 512 and 256 dimensions against the full representation on the searches the product promises to answer. Both intermediate settings retain much closer benchmark scores while reducing raw vector requirements. The smallest option should earn its place through acceptable results, rather than win automatically because it is smallest.

A 13.36-point MMEB decline is a difference in benchmark score, not a percentage of customers receiving a wrong answer. Nor does this table measure the quantized Pixel runtime behind Google’s memory figures: these evaluations use the full-precision checkpoint. The evidence supports a reason to test intermediate settings, not a universal verdict against 128 dimensions.

A second compression lever moves some cost elsewhere

Dimensional truncation is not the only mechanism available to reduce vector memory. Practitioners frequently turn to scalar or binary quantization, which decreases the number of bits allocated to each individual vector component rather than discarding dimensions.

Qdrant, a vector database vendor with early access to the model, tested five BEIR text datasets: SciFact, NFCorpus, ArguAna, SCIDOCS and FiQA. Its 768-dimensional, 1-bit TurboQuant configuration used 104 bytes per vector against 3,072 bytes for float32, retaining 99 percent of baseline retrieval quality as measured by average nDCG@10. That is roughly thirtyfold lower vector RAM.

At 256 dimensions and 40 bytes per vector, retained quality was 94.5 percent with fourfold candidate oversampling and original-vector rescoring; without rescoring it was 88.1 percent. These are text tests by a commercially interested external vendor. The reported vector-memory saving excludes graph indexes, payloads and originals kept for rescoring. It establishes neither multimodal quality nor a phone’s total memory saving.

Rescoring changes what “compressed” means for the application. A coarse search chooses candidates from the smaller representation, then compares those candidates using retained originals. If those originals sit on disk, fetching them adds reads. The benefit is a smaller working representation, while the fuller information still has to live somewhere. Qdrant’s documentation describes asymmetric quantization, with compressed stored vectors and full-precision queries; compressing the collection need not mean treating every query the same way.

This is a useful alternative to throwing away more dimensions, but it also creates a product choice. How much storage can the application reserve for originals? How many candidates should it reconsider? A developer may accept that arrangement for a text archive. A media product needs evidence about its own visual and audio queries before treating the same byte count as a reliable setting. Google’s local demos do not establish that they use Qdrant or TurboQuant.

A collection grows differently from its model

The collection arithmetic explains why a small model is only the beginning. Consider an illustrative float32 index, before choosing any database implementation.

In standard IEEE 754 float32 precision, every vector dimension requires four bytes of storage. A single 768-dimensional vector therefore occupies 3,072 bytes. A 256-dimensional vector requires 1,024 bytes, and a 128-dimensional vector requires 512 bytes.

If an application indexes 100,000 units, storing those representations at full 768-dimensional float32 precision requires approximately 307.2 megabytes of raw vector capacity. At one million items, that figure expands to approximately 3.072 gigabytes. Slicing those representations to 128 dimensions reduces the raw vector footprint for one million items to roughly 512 megabytes.

These calculations account solely for raw vector floating-point arrays. They do not include the auxiliary memory consumed by nearest-neighbor index structures (such as HNSW graphs or inverted file tables), document identifiers, payload metadata, operating system memory allocator overhead, or the source media files themselves.

An indexed unit can be a passage or a media segment, so vector count need not match file count. EmbeddingGemma 2 operates across an 8,192-token input context window, but its default ingestion parameters assign substantial token weights to non-text media. In its default configuration, an image consumes 280 tokens. Audio input consumes 25 tokens per second. Video, sampled by default at one frame per second, consumes 140 tokens per frame.

At those defaults, a ten-minute video sampled once a second would present 600 frames and 84,000 vision tokens, beyond a single input window. The model card’s approximately 58-frame limit assumes the default frame budget and no accompanying text; sampling and vision-token budgets can be changed. There is no fixed number of vectors that every ten-minute video must produce.

The more revealing choice is what a search result should identify. A vector for a short clip can help find the clip. A user asking for the moment someone puts a red bag on a table may need a narrower segment. Splitting a recording into smaller units can make those moments separately retrievable, at the cost of more embeddings and more entries. Coarser sampling can also omit the brief event the user wants. Neither decision is resolved by the model’s weight footprint.

A hypothetical application could keep broad representations for browsing and narrower ones for selected recordings. That would reserve detailed indexing for the content where it adds value, with additional complexity in managing the two levels. The right balance depends on whether people are looking for a document, a scene or a spoken phrase. The searchable unit is part of the product design, not just a storage setting.

Local retrieval is available; the complete workflow still matters

Google’s launch demonstrations give this design discussion a concrete starting point.

In its technical deployment documentation, the Google AI Edge and Android ML engineering teams demonstrated applications using EmbeddingGemma 2. In reference implementations such as InstantMediaSearch and VideoMomentsFinder, query and corpus embeddings are generated on-device and managed through a local SQLite database with cosine similarity used to retrieve matching content. On the desktop, Google showcased Foresight on Mac, an experimental local workspace interface that pairs EmbeddingGemma 2 embeddings with a local Gemma 4 model for file retrieval and conversational synthesis.

The distinction between an embedder and a retriever matters here: generating a useful vector and finding a useful result are separate operations. Google’s MediaPipe APIs cover embedding and retrieval tasks, including preprocessing support. The demos show a local workflow; they do not supply a measured resource budget for every collection an application might import.

Google says an ML Kit integration is coming in the weeks after launch, including NPU acceleration when available. That future path may simplify Android integration, but a team choosing a shipping configuration today must distinguish the available tools from the announced service. The release does not yet establish sustained battery or thermal performance for a user’s full indexing job.

EmbeddingGemma 2 uses task-specific text prefixes, including different prefixes for search queries and indexed passages. The model card says omission still works but reduces precision. An application can spend carefully on vector storage and still lose useful results through a mismatch in how it prepares queries and documents.

Local inference can keep that operation off a remote service. What happens next determines whether the whole workflow stays offline. If retrieved clips go to a cloud model for a written summary, that later stage needs a connection and sends the selected material away. A search interface that simply returns local results does not require generation at all. Adding a local generative model introduces another resource demand rather than completing a step inherent to embeddings.

Choose the searches worth keeping

EmbeddingGemma 2’s strongest proposition is a shared search space for several kinds of local content. A person could describe an image they cannot remember naming, or look for a relevant section of a recording without first producing a separate text transcript. Keeping those representations and searches local gives developers a concrete way to reduce dependence on a remote retrieval service.

That promise makes the quality tradeoff more important. A product whose purpose is finding images and audio should not adopt its storage configuration solely from the text benchmark. Equally, a small document collection may have little reason to pay the complexity of original-vector rescoring just because an external test reports a dramatic compression ratio. The workload should decide which saving is valuable.

I would judge the release by whether an application can keep its intended collection searchable at acceptable quality, with enough room for the rest of the workflow. Google’s intermediate dimensions offer a credible starting point; its smallest vectors and Qdrant’s compressed text tests show two different ways to save space, with different costs. The useful achievement is a local search product that keeps answering the questions people ask as its collection grows. Fitting the model is the first requirement.