Skip to content
ContentLora

    Tip: press / anywhere to search.

    Explainer

    How embeddings work: turning words and images into numbers

    An embedding is a list of numbers, a vector, that represents a piece of data such as a word, sentence or image in a lower-dimensional space.[1] The distance between two embeddings can be calculated and read as a measure of how similar the model considers them, which is what lets software find related items by meaning rather than by matching keywords.[2][3] Embeddings are stored in vector databases and used for semantic search, classification, clustering and retrieval for chatbots.[4][3][5]

    Editor reviewedUpdated Explainer19 claims · 100% primaryArtificial intelligence, Frontier AI

    Embeddings are lists of numbers that represent words, images and other data so that a computer can compare them. For the wider context, see how machine learning works and how transformers work.

    The problem embeddings solve

    Computers need numbers, and the simplest way to turn a list of items into numbers wastes a lot of space. Google’s machine learning course uses a dataset of 5,000 popular dishes: written as one-hot vectors, each dish becomes a list of 5,000 entries with a single 1, and the more weights a model has to learn from such large inputs, the more data it needs to train.[6]

    An embedding is the alternative. It is a vector representation of the data, found by projecting the high-dimensional starting data into a lower-dimensional space.[1] Instead of 5,000 mostly empty slots, each item gets a shorter list of numbers.[1]

    Distance as similarity

    What makes embeddings useful is that position carries meaning. In an embedding, the distance between any two items can be calculated and interpreted as a measure of how similar the model considers them.[2] A common measure is cosine similarity, which Google’s Gemini documentation recommends because it looks at the direction of the vectors rather than their length; it runs from -1 for opposites to 1 for the most similar.[7]

    Real embedding spaces have many dimensions. Google says word embeddings often use 256, 512 or 1,024, and the individual dimensions are rarely easy to label.[8] Often the arrangement is not interpretable by humans at all.[9] Embeddings also depend on what they were trained for: Google’s course notes they are usually specific to a task and differ when the task differs.[10]

    From word2vec to contextual embeddings

    Word embeddings rest on a simple idea. Models trained to predict the context of a word assume that words appearing in similar contexts are related in meaning.[11] The 2013 word2vec paper reported learning high-quality word vectors from a 1.6-billion-word dataset in less than a day, and word2vec gives each word a single fixed vector, which Google’s course calls a static embedding.[12]

    Static embeddings can capture relationships: Google’s glossary says word2vec can place cow and bull about as far apart as ewe and ram.[13] Contextualized embeddings go further, for example recognizing that English speakers sometimes use “cow” loosely to mean either cow or bull.[13] In a Transformer, each self-attention layer transforms one sequence of token embeddings into another.[14]

    Embeddings also work at the level of whole sentences. The 2019 Sentence-BERT paper produced sentence embeddings that can be compared with cosine similarity, cutting the time to find the most similar pair among 10,000 sentences from about 65 hours with BERT to about 5 seconds.[15]

    What embeddings are used for

    Google says its Gemini embeddings can be used for semantic search, classification and clustering.[3] Its gemini-embedding-2 model maps text, images, video, audio and documents into one shared space.[16] Its embedding models are trained so that the 3,072-number vectors can be shortened to save storage while keeping their usefulness.[17]

    Search is a common use. A 2020 paper showed that passage retrieval for question answering could rely on learned embeddings alone and beat a strong keyword-based BM25 system by 9 to 19 percentage points in top-20 accuracy.[18] This is the retrieval step in retrieval-augmented generation, where documents are split into chunks, turned into embeddings and searched by similarity to a user’s question.[5]

    Storing and searching embeddings

    At scale, embeddings live in vector databases, which store, index and retrieve large collections of vectors; libraries such as Faiss are dedicated to vector similarity search, the core function of these databases.[4] Search based only on embeddings can miss exact terms such as an error code, so many systems pair it with keyword matching in what is called hybrid search.[19] For how search engines and recommendation feeds use similar ideas, see how search engines work and how recommendation algorithms work.

    Developing in AI & Computing

    Questions readers ask

    What is an embedding in AI?

    A vector representation of data. A model finds embeddings by projecting high-dimensional data, such as a vocabulary of thousands of words, into a lower-dimensional space where each item is a list of numbers.[1]

    Why are similar things close together in an embedding?

    Training arranges items so that the distance between them reflects how related the model finds them. For words, models trained to predict a word's context assume that words appearing in similar contexts are related in meaning.[2][11]

    How many dimensions does an embedding have?

    Far more than three. Google says word embeddings often use 256, 512 or 1,024 dimensions, and its Gemini embedding models output 3,072 by default, which can be truncated to save storage.[8][17]

    What is cosine similarity?

    A common way to compare two embeddings. It measures the angle between the vectors rather than their length, giving a value from -1 (opposite) to 1 (most similar).[7]

    What is a vector database?

    A database built to store, index and search large collections of embeddings. Libraries such as Faiss are dedicated to vector similarity search, the core function of these databases.[4]

    Can images and text share one embedding space?

    Yes. Google says its gemini-embedding-2 model maps text, images, video, audio and documents into one shared embedding space, which allows searching across types of media.[16]

    Sources

    · 19 claims from 5 publishers

    Each numbered claim is a statement we checked against the sources listed with it. Tap a number in the text to see its sources in place.

    1. [1]

      Google's machine learning course defines an embedding as a vector representation of data, usually found by projecting high-dimensional data into a lower-dimensional space. confirmedas of 2026-11-14

    2. [2]

      In an embedding, the distance between two items can be calculated and read as a measure of how similar the model considers them. confirmedas of 2026-11-14

    3. [3]

      Google says embeddings from its Gemini embedding models can be used for tasks such as semantic search, classification and clustering. confirmedas of 2026-11-14

    4. [4]

      Vector databases store, index and retrieve large collections of embeddings, and the Faiss library is dedicated to vector similarity search, a core function of such databases. confirmedas of 2026-11-14

      • Embeddings (Gemini API) · Google AI for Developers · Primary source · Storing embeddings (retrieved 2026-11-14)
      • The Faiss library · arXiv · Primary source · 2024-01-16 · Abstract (retrieved 2026-11-14)
    5. [5]

      Anthropic describes a typical RAG pipeline as splitting a knowledge base into chunks of usually no more than a few hundred tokens, converting them into embeddings stored in a vector database, and at query time finding the chunks most similar to the query. confirmedas of 2024-09-19

    6. [6]

      Google's course illustrates the problem embeddings solve with a 5,000-item food dataset, where each one-hot vector has 5,000 entries and more weights mean more data is needed to train. confirmedas of 2026-11-14

    7. [7]

      Google's Gemini documentation calls cosine similarity a good measure for embeddings because it looks at direction rather than magnitude, with values from -1 (opposite) to 1 (most similar). confirmedas of 2026-11-14

      • Embeddings (Gemini API) · Google AI for Developers · Primary source · Semantic similarity (retrieved 2026-11-14)
    8. [8]

      Real embedding spaces have far more than three dimensions; Google says word embeddings often use 256, 512 or 1,024, and the individual dimensions are rarely easy to interpret. confirmedas of 2026-11-14

    9. [9]

      Google's course says the relative positions of items in an embedding space are often not interpretable by humans. confirmedas of 2026-11-14

    10. [10]

      Google's course says embeddings are usually specific to the task they were trained for and differ when the task differs. confirmedas of 2026-11-14

    11. [11]

      Models trained to predict the context of a word assume that words appearing in similar contexts are related in meaning. confirmedas of 2026-11-14

    12. [12]

      The 2013 word2vec paper reported learning high-quality word vectors from a 1.6-billion-word dataset in less than a day; word2vec gives each word a single global embedding, which Google's course calls a static embedding. confirmedas of 2013-01-16

    13. [13]

      Google's glossary says older embeddings such as word2vec can place cow and bull about as far apart as ewe and ram, while contextualized embeddings can also recognize that people sometimes use "cow" to mean either. confirmedas of 2026-11-14

      • Machine Learning Glossary · Google for Developers · Primary source · Glossary, contextualized language embedding (retrieved 2026-11-07)
      • Machine Learning Glossary · Google for Developers · Primary source · Glossary, contextualized language embedding (retrieved 2026-11-07)
    14. [14]

      A self-attention layer transforms a sequence of embeddings, such as token embeddings, into another sequence of embeddings, and self-attention is one of the main building blocks of Transformers. confirmedas of 2026-11-07

    15. [15]

      The 2019 Sentence-BERT paper produced sentence embeddings that can be compared with cosine similarity, cutting the time to find the most similar pair among 10,000 sentences from about 65 hours with BERT to about 5 seconds. confirmedas of 2019-08-27

    16. [16]

      Google says its gemini-embedding-2 model is the first multimodal embedding model in the Gemini API, mapping text, images, video, audio and documents into one shared embedding space. confirmedas of 2026-11-14

    17. [17]

      Google's Gemini embedding models output 3,072-dimensional embeddings by default and are trained with Matryoshka Representation Learning, so the vectors can be truncated to a smaller size to save storage. confirmedas of 2026-11-14

      • Embeddings (Gemini API) · Google AI for Developers · Primary source · Controlling embedding size (retrieved 2026-11-14)
      • Embeddings (Gemini API) · Google AI for Developers · Primary source · Controlling embedding size (retrieved 2026-11-14)
    18. [18]

      A 2020 paper showed that passage retrieval for question answering could use learned dense embeddings alone, beating a strong BM25 keyword system by 9 to 19 percentage points in top-20 retrieval accuracy. confirmedas of 2020-04-10

    19. [19]

      Embedding search can miss exact terms such as an error code that keyword matching like BM25 finds, so many RAG systems combine the two, called hybrid search, and add a re-ranker to put the most relevant results first. confirmedas of 2026-11-14

    Revision history (1)
    1. Page created.

    Created Nov 14, 2026. Last reviewed by an editor on Nov 14, 2026. Next scheduled review: May 14, 2027.

    Cite this page

    "How embeddings work: turning words and images into numbers." ContentLora, updated Nov 14, 2026. https://contentlora.com/explain/how-embeddings-work

    Spotted an error? Suggest a correction or emailcorrections@contentlora.com.