Measure consistently. Compare carefully.
Make model evaluations useful by preserving the conditions behind every number.
Pin the inputs
Record the model artifact, runtime version, prompt set and generation settings. Use the same protocol when comparing candidates.
Separate the measurements
Report startup, time to first token, generation throughput and peak memory separately. State how each number was measured and whether a warm-up was excluded.
Keep context in context
A runtime's configured context limit does not prove that a full-length workload was tested. Record actual input and output lengths, and evaluate output quality alongside performance.