Embedding

Data
About 1 min read

A technology that converts unstructured data, such as text and images, into high-dimensional numerical vectors that preserve meaning so that AI can process them. Beyond simple numerical conversion, it mathematically represents contextual relationships and similarity between data points.

Also known as
EmbeddingVector Embedding

Detailed explanation

Embedding is the process of mapping data to coordinates in a multidimensional vector space, making 'semantic similarity' calculable. For example, 'apple' and 'pear' are positioned close to each other in space, whereas 'apple' and 'car' are placed far apart. Recently, high-performance yet cost-efficient models like OpenAI's 'text-embedding-3' or Google's 'Gemini Embedding 2' have become mainstream. In particular, modern models that support the 'Matryoshka' representation learning technique can freely reduce vector dimensions with minimal performance degradation, saving storage costs. This is the key factor determining the retrieval quality in Retrieval-Augmented Generation (RAG) and serves as the infrastructure for recommendation systems and semantic search.

Why it matters in tool selection

The performance of the embedding model directly determines the intelligence of the AI service. Using an inappropriate model in Retrieval-Augmented Generation (RAG) causes 'retrieval failure,' where the system cannot find the information needed for a response. Additionally, while larger vector dimensions improve accuracy, they increase database storage costs and search latency; therefore, selecting a cost-effective model suited to your purpose is key to operational efficiency.

What to look for when choosing an embedding model

  • Check the Retrieval score on the Massive Text Embedding Benchmark (MTEB)
  • Maximum supported input token length (modern models usually support 8K to 32K or more)
  • Whether it supports dimension reduction (Matryoshka Representation Learning)
  • Whether its performance has been verified on multilingual data, including Korean

Real-world examples

When a user asks, 'Check the details of the deposit I made yesterday,' the AI converts this sentence into a vector. It then finds the contextually closest (most similar) data among numerous financial record vectors stored in a vector database to construct the answer.

Confusing terms

Tokenization

A simple preprocessing step that breaks a sentence down into machine-readable units (tokens).

Embedding

An advanced process that infuses split tokens with deep context and meaning, mapping them to mathematical coordinates.

Related terms

Vector DatabaseRAGSemantic Search