Independently reproducedpublished Reproduction

Independent reproduction on RTX 4070 Ti

Reproduced the published suite within the expected score and speed range.

Orion Coder 7B @mateo Jul 19, 2026

Manifest JSON Reproduce this Run

Speed

67.3 tok/s

Time to first token

201 ms

Peak memory

5.9 GB

Coding suite

73.9

Useful

103

4 reproductions

Reproduction command

Use the exact revision and inspect the manifest before execution.

modelverse reproduce run_orion_m3max_coding --runtime llama.cpp

Environment and configuration

The comparison context that makes the result interpretable.

Hardware

MachineRTX 4070 Ti · 12 GB

CPUAMD Ryzen 9 7900

AcceleratorNVIDIA GeForce RTX 4070 Ti

VRAM12 GB

System memory32 GB

Operating systemUbuntu 26.04

Architecturex64

Compute APICUDA 13.0

Configuration

Runtimellama.cpp b5881

Model revisiond4f7b92

QuantizationQ4_K_M

Context length8192

Temperature0

Seed42

Batch size1

Portable Run manifest

Version 1.0 of the Apache-2.0 licensed Modelverse Run schema.

{
  "schemaVersion": "1.0",
  "id": "run_orion_4070ti_reproduction",
  "createdAt": "2026-07-19T14:10:00.000Z",
  "model": {
    "provider": "huggingface",
    "repository": "open-lab/orion-coder-7b",
    "revision": "d4f7b92",
    "quantization": "Q4_K_M",
    "licenseSnapshot": "Apache-2.0"
  },
  "runtime": {
    "name": "llama.cpp",
    "version": "b5881",
    "command": "modelverse reproduce run_orion_m3max_coding --runtime llama.cpp"
  },
  "hardware": {
    "os": "Ubuntu 26.04",
    "architecture": "x64",
    "cpu": "AMD Ryzen 9 7900",
    "systemMemoryMb": 32768,
    "accelerators": [
      {
        "vendor": "NVIDIA",
        "model": "NVIDIA GeForce RTX 4070 Ti",
        "count": 1,
        "memoryMb": 12288,
        "driverVersion": "590.44",
        "computeApi": "cuda"
      }
    ]
  },
  "task": {
    "type": "coding",
    "evaluationSuite": "modelverse-local-coding-v1",
    "datasetRevision": "6ef21a9"
  },
  "configuration": {
    "contextLength": 8192,
    "maxGeneratedTokens": 512,
    "temperature": 0,
    "seed": 42,
    "batchSize": 1,
    "quantization": "Q4_K_M"
  },
  "metrics": {
    "timeToFirstTokenMs": 201,
    "tokensPerSecond": 67.3,
    "peakMemoryMb": 6041.6,
    "peakVramMb": 6553.6,
    "modelLoadTimeMs": 1120,
    "scores": {
      "Coding suite": 73.9
    }
  },
  "evidence": {
    "machineCaptured": true,
    "redactionsApplied": [],
    "artifactUrls": []
  }
}

Notes and interpretation

Independent demo reproduction using the same suite and model revision.

Evaluation suite

modelverse-local-coding-v1

dataset revision 6ef21a9

Other Runs for Orion Coder 7B

Standardized evaluation2 months ago

Orion Coder 7B on M3 Max · local coding suite

by @alice · MLX 0.29.2

Speed

42.8 tok/s

TTFT

318 ms

Peak

6.1 GB

Useful

187

Machine-captured2 months ago

Orion Coder 7B with Ollama inside 16 GB

by @chen · Ollama 0.12.1

Speed

31.4 tok/s

TTFT

480 ms

Peak

7.2 GB

Useful

149

Environment and configuration

The comparison context that makes the result interpretable.

Hardware

MachineRTX 4070 Ti · 12 GB

CPUAMD Ryzen 9 7900

AcceleratorNVIDIA GeForce RTX 4070 Ti

VRAM12 GB

System memory32 GB

Operating systemUbuntu 26.04

Architecturex64

Compute APICUDA 13.0

Configuration

Runtimellama.cpp b5881

Model revisiond4f7b92

QuantizationQ4_K_M

Context length8192

Temperature0

Seed42

Batch size1

{ "schemaVersion": "1.0", "id": "run_orion_4070ti_reproduction", "createdAt": "2026-07-19T14:10:00.000Z", "model": { "provider": "huggingface", "repository": "open-lab/orion-coder-7b", "revision": "d4f7b92", "quantization": "Q4_K_M", "licenseSnapshot": "Apache-2.0" }, "runtime": { "name": "llama.cpp", "version": "b5881", "command": "modelverse reproduce run_orion_m3max_coding --runtime llama.cpp" }, "hardware": { "os": "Ubuntu 26.04", "architecture": "x64", "cpu": "AMD Ryzen 9 7900", "systemMemoryMb": 32768, "accelerators": [ { "vendor": "NVIDIA", "model": "NVIDIA GeForce RTX 4070 Ti", "count": 1, "memoryMb": 12288, "driverVersion": "590.44", "computeApi": "cuda" } ] }, "task": { "type": "coding", "evaluationSuite": "modelverse-local-coding-v1", "datasetRevision": "6ef21a9" }, "configuration": { "contextLength": 8192, "maxGeneratedTokens": 512, "temperature": 0, "seed": 42, "batchSize": 1, "quantization": "Q4_K_M" }, "metrics": { "timeToFirstTokenMs": 201, "tokensPerSecond": 67.3, "peakMemoryMb": 6041.6, "peakVramMb": 6553.6, "modelLoadTimeMs": 1120, "scores": { "Coding suite": 73.9 } }, "evidence": { "machineCaptured": true, "redactionsApplied": [], "artifactUrls": [] } }