Taken at Hack The North 25
Take notes on cohere-ai/htn-2025-techtalk
Define: Train-Time Compute
- The compute required to train the model
Define: Test-Time Compute
- The compute required to run the model and test it
How do we increase Test-Time compute?
- We can “vertically” scale, increase number of token, and then get more info from it
- We can “horizontally” scale, increase number of times we ask the model, and thus have more responses
Vertically Scaling of TTC
- Prompt the model to think longer,
Verifiers
- Just for this talk, a verifier is an entity that assigns a label of correctness to a single LLM trajectory.
Synthesis appraoch methods
- Ground-truth programmatic verification
- Take the majority answer (maj@k)
- Ask another LLM (or a panel of LLMs) to choose the best one (most complex and hand-wavy)
Best ML Practices
Before Anything:
- Define your verifier
- Define an evaluation set
- consistent of prompts, defined grouth truth, etc.
- larger the evaluation set the better
Toy Problem
htn-2025-techtalk/toy_problem.ipynb at master · cohere-ai/htn-2025-techtalk
We can scale horizontally and find similar gains without any training! By simply making a validator, and then upping the number of responses you prompt.
QnA
Define: Sampling Parameters eg.
- Temperature
- Top-p
- Top-k
There is a set of “optimal parameters” that the model was trained on and are to be used to vertically scale in the best way.