Cold Start

Technical terms
About 1 min read

The response latency that occurs when an AI system is launched from an inactive state, or the phenomenon in recommendation systems where accurate results cannot be provided to new users due to a lack of data.

Also known as
Cold BootInitial Startup LatencyData Sparsity Problem

Detailed explanation

Cold start must be understood in two core contexts. First, from an infrastructure perspective, it refers to the latency that occurs in a serverless environment (such as AWS Lambda or GCP Functions) when an idle model instance is allocated computing resources and loads its code and runtime to process the first request. This directly impacts the user experience of AI services where real-time response is critical. Second, from a data perspective, it refers to the 'data sparsity' state where a recommendation system cannot derive personalized results because there is no history of past interactions for a new user or item. When adopting AI tools, it is essential to check if they offer provisioning features to minimize such infrastructure latency, or hybrid recommendation logic (utilizing metadata, etc.) to fill data gaps.

Why It Matters in Tool Selection

Cold start determines the 'first impression' of a service. If model loading is slow, users leave, and if irrelevant recommendations are provided to new users, service trust drops. Therefore, for services with frequent large-scale traffic or influx of new users, the ability to control cold starts is a key criterion for tool selection.

What to Look For

  • Does it provide 'always-warm' or provisioned concurrency capabilities in a serverless environment?
  • Does backup logic based on popularity or metadata activate when new users/items occur?
  • Is runtime lightweighting and dependency optimization supported during model deployment?

Example

Showing 'most popular content' first to a newly registered user because they have no viewing history is a response to a data cold start, whereas an image generation AI model that was dormant during dawn hours taking over 10 seconds to respond to its first request is an example of an infrastructure cold start.

Related terms

LatencyServerless