EmbeddingGemma 2: multimodal search and setup

A Colab walkthrough of EmbeddingGemma 2 for photo, voice, video and PDF search, with setup fixes and measured retrieval limitations.

Player not loading? Watch on YouTube

This walkthrough explains EmbeddingGemma 2 and tests multimodal retrieval in a hosted Google Colab T4 notebook. The presenter describes modality encoders, a shared Gemma 4 backbone and mean pooling into 768-dimensional vectors. The contrastive-training explanation draws on related Gemini embedding research because, according to the speaker, this model did not yet have a technical report.

The notebook uses Sentence Transformers. Setup notes cover adding torchcodec for video decoding and avoiding float16 because activation overflow can produce invalid embeddings. The presenter uses float32 and recommends bfloat16 on newer GPUs. Phone deployment is discussed, but the practical experiments run in Colab.

The demos cover multilingual photo queries, voice queries without transcription, video search and PDF page retrieval without OCR. Across 1,000 photos and 5,000 caption queries, the presenter reports about 86% top-one recall and 97% top-five recall. A separate test retrieves the correct environmental sound first in 24% of 50 queries.

Reducing photo vectors to 256 dimensions gives 84% top-one recall in this run; 128 dimensions gives 75%. A text-only component queries the photo index built with the full model. The speaker advises testing similarity thresholds and environmental audio on the intended data.