Temperature

AI concepts
About 1 min read

A value that controls the degree of randomness when an LLM selects the next word, determining the consistency and creativity of the output.

Also known as
TemperatureSampling temperatureTemperature parameter

Detailed explanation

Temperature is a value that determines how flat the probability distribution will be when an LLM selects the next token, controlling output randomness. Low values (e.g., 0 to 0.3) focus on the highest probability tokens, making responses consistent and predictable, which is suitable for fact-based tasks. High values (e.g., 0.8 to 1.2) allow less probable tokens to be selected, yielding diverse and creative expressions, though at a higher risk of going off-topic or hallucinating facts. Because outputs can vary significantly based on temperature even for the same prompt, adjusting it to fit the nature of the task is crucial. It is commonly used alongside other sampling options like top-p (nucleus sampling).

Why It Matters in Tool Selection

If AI tool outputs are inconsistent or overly predictable, the temperature setting is often the cause. Whether a tool exposes temperature settings to the user or provides task-specific presets determines how effectively you can control the output as intended. A low temperature is ideal for tasks requiring factual accuracy, while a high temperature is suitable for brainstorming ideas.

Key Considerations

  • Can users adjust temperature or randomness settings?
  • Does it provide or guide users toward a low-temperature mode for fact-based tasks?
  • Can you also adjust other sampling options like top-p?
  • Does it support tasks requiring reproducibility of results for the same input (such as a fixed seed)?

Real-World Examples

A policy-guiding chatbot should have its temperature set close to 0 to consistently return the same factual answers, whereas an ad copy brainstorming tool uses a higher temperature to generate diverse expressions. Since a single value can alter the output style of the exact same model, offering settings tailored to the application is crucial.

Related terms

LLMPrompt EngineeringHallucination