Your models. Your hardware.
Prepare a local model runtime and make your measurements reproducible.
Choose a runtime
Ollama and llama.cpp are local inference tools. Follow the project's installation guide and the model publisher's instructions for a compatible artifact.
Check the configuration
Record the model artifact, quantization, runtime version, available memory and context length. Keep the configuration fixed while comparing results.
Measure the work that matters
Use representative inputs and repeat each measurement. Keep startup time, time to first token and generation throughput separate. Review outputs for quality as well as speed.