temperature
A number that sets how strongly a language model favours its highest-scoring next word. Low values keep it on the favourite, high values make it more likely to choose from words with lower scores.
Example
Asked to finish A cowboy doll is jealous when a new ___, the model gives every word it knows a score. These three score highest.
spaceman 4.0
toy 3.0
friend 2.0Those scores are turned into percentages. Before that happens, each one is divided by the temperature. Dividing by a small number stretches the gaps apart. Dividing by a large one squashes them together.
T = 0.5 spaceman 86.7% toy 11.7% friend 1.6%
T = 1.0 spaceman 66.5% toy 24.5% friend 9.0%
T = 2.0 spaceman 50.6% toy 30.7% friend 18.6%The model then picks one word, using those percentages as the odds at every temperature.
Turning the temperature up does not switch randomness on. It only shortens the odds on the words further down the list.
Dividing every score by the same number cannot lift one word past another, so the order remains consistent.
At 0.1, a top-scoring word could hold a 99.995% probability, so a model with low temperature appears to provide the same answers, consistently.
Lessons that use this term
The lesson and course links below open in a new tab.
No published lesson uses this term yet.