Parameter

Data
About 1 min read

Internal variables (weights and biases) that an AI model saves through learning. Larger numbers allow the model to learn complex patterns, but also increase computational and memory costs.

Also known as
ParameterParameterWeights

Detailed explanation

Parameters are the sum of weights and biases multiplied by and added to input data within an artificial neural network. They are units of 'internal knowledge' that an AI model optimizes on its own during training, and the model's scale is typically represented using the unit 'B' (Billion). The larger the number of parameters, the more the model can learn correlations in vast datasets, enabling higher-level reasoning and creative tasks. However, as parameters increase, the computational resources (VRAM) and inference time (latency) required to run the model also increase. Therefore, rather than simply choosing the largest model, it is crucial to select a scale optimized for your specific purpose and operating environment.

Why it matters in tool selection

The number of parameters is a metric that determines the cost-efficiency and performance when adopting AI. Large models with 70B or more parameters are advantageous for complex logical reasoning, but are expensive to operate. On the other hand, 7B to 13B models (SLMs) well-trained on domain-specific data can deliver performance on par with giant models in specific tasks at a fraction of the cost.

What to look for

  • Can your GPU memory (VRAM) handle the parameter size of the selected model?
  • Are you using an excessively large parameter model for simple tasks like text classification or summarization?
  • Is it a modern architecture model with outstanding benchmark (e.g., MMLU) performance efficiency relative to its parameter count?

Examples

The Llama 3 8B model has a small number of parameters, making it runnable on consumer PCs and suitable when fast response times are needed. In contrast, GPT-4 (estimated at trillions) or the Llama 3 400B+ model are used when advanced legal/medical expertise or complex programming support is required.

Confusing terms

Hyperparameter

Control values set directly by humans before training starts, rather than learned by the model itself (e.g., learning rate, batch size).

Token

Unlike parameters, which represent the size of a model's intelligence, this is a quantitative unit of data processed or generated by the model at one time.

Related terms

Fine-tuningMoE (Mixture of Experts)