Serverless

Infrastructure
About 1 min read

A cloud computing model where you run code or services without managing infrastructure, featuring automatic scaling and billing based on actual usage (number of invocations, execution time).

Also known as
ServerlessServerless Computing

Detailed explanation

Serverless is an execution model that helps developers focus solely on the core logic of their applications without worrying about provisioning, managing, or scaling servers. It does not mean there are no servers; rather, the cloud service provider fully manages the infrastructure. Recently, in the AI field, 'serverless GPUs'—which spin up only when there are requests and reduce costs to 'zero' when idle, instead of running expensive GPUs continuously—have emerged as a core technology. This dramatically cuts operating costs for AI services with irregular traffic and enables rapid market entry without the burden of infrastructure management.

Why it matters for AI tool selection

AI models require high-spec GPUs, resulting in very high fixed costs when run continuously. Choosing a serverless model helps save on idle GPU costs during off-peak hours and automatically responds to sudden traffic spikes, making it essential for early-stage cost optimization in startups.

What to check when choosing tools

  • Cold Start: Is the latency that occurs when processing the first request from an idle state acceptable given the size of the AI model?
  • Maximum Execution Time: For generative AI models requiring long inference times, does it hit the platform's timeout limits?
  • GPU Availability: Can high-performance GPU resources like NVIDIA A100 or H100 be provisioned instantly when needed?

Real-world use cases

It is primarily used for APIs like Stable Diffusion, which spin up to generate images only when a user inputs text, or custom LLM chatbot backends that remain idle and generate responses only when a question is received.

Infrastructure model comparison

IaaS (Infrastructure as a Service)

Directly purchasing/managing virtual servers. Offers high flexibility but comes with high operational overhead and idle costs.

Serverless (FaaS/GPU)

Zero operational management and pay-as-you-go pricing. Capable of scaling to zero, though initial latency (cold starts) may occur.

Related terms

Cold Start