What every section and number below means: reading-reports.md.
Cost model scaled to this device from one probe point: bidiag x1.00, block x0.25, cpu x0.08, jacobi x0.10, qr x0.21, qrblock x0.25 (1.00 is an M1).
Machine state: load 2.7/18 at the start, load 6.9/18 at the end; power mains.
Probe point after the sweep relative to before it: block x0.98, cpu x0.99, jacobi x0.99, qr x0.98, qrblock x1.00 (stable).
Generated by tuning/tune_svd.py from 266 (shape, batch) points, 41 shapes with M >= N, batch in [1, 4, 16, 64, 256, 1024, 4096], six backends (and the CPU and bidiag again for singular values alone), two or more passes, min-of-repeats.
Row for kTuned[] in src/svd.mm:
// device, GPU cores, qr_min_rows, qr_min_k, block_min_k, block_min_k_batched, block_min_batch, gpu_max_k, gpu_min_batch_times_k, gpu_min_batch, bidiag_min_k, values_bidiag_min_k, bidiag_max_batch, values_bidiag_max_batch
{"Apple M5 Pro", 20, 512, 32, 192, 64, 64, 8, 4096, 1, 1024, 2048, 1, 1},
To try it without rebuilding:
SVD_QR_MIN_ROWS=512 SVD_QR_MIN_K=32 SVD_BLOCK_MIN_K=192 SVD_BLOCK_MIN_K_BATCHED=64 SVD_BLOCK_MIN_BATCH=64 SVD_GPU_MAX_K=8 SVD_GPU_MIN_BATCH_TIMES_K=4096 SVD_GPU_MIN_BATCH=1 SVD_BIDIAG_MIN_K=1024 SVD_VALUES_BIDIAG_MIN_K=2048 SVD_BIDIAG_MAX_BATCH=1 SVD_VALUES_BIDIAG_MAX_BATCH=1
The policy in effect on this device came from tuned:Apple M5 Pro. Against the best measured backend at every point the fitted rule scores 1.0019 geometric-mean regret, worst 1.25x, 3 of 266 points losing more than 10%, and 1.000x the oracle’s total time.
Scored against the best GPU backend at each of the 266 points, as if there were no CPU: this is the rule a forced-GPU call (SVD_DEVICE=gpu) follows, and what a GPU with more cores will lean on. Two independent choices. Precondition with QR iff the long side is at least qr_min_rows, the short side at least qr_min_k, and the long side at least twice the short one. Block kernel iff the short side is at least block_min_k, or at least block_min_k_batched in a batch of block_min_batch or more.
| rule | geomean regret | worst | >10% | total time / oracle | est. picks |
|---|---|---|---|---|---|
| policy in effect (‘512’, ‘32’, ‘192’, ‘64’, ‘64’) | 1.0299 | 1.73x | 28 | 1.016 | 0 |
| fitted (‘512’, ‘32’, ‘192’, ‘64’, ‘64’) | 1.0299 | 1.73x | 28 | 1.016 | 0 |
Without a batch term, 2 of 588 combinations are within 0.5% of the best geomean: qr_min_rows 512 .. 512, qr_min_k 32 .. 32, block_min_k 96 .. 128.
With the batch term adopted (below), 2 combinations are within 0.5% of the best: block_min_k 192 .. 256, block_min_k_batched 64 .. 64, block_min_batch 64 .. 64. The curves vary one constant around the chosen combination.
xychart-beta
title "Regret by block_min_k"
x-axis "block_min_k" [32, 48, 64, 96, 128, 192, 256, 384, 512, 768, 1024, none]
y-axis "geometric-mean regret" 1.0 --> 1.25
line [1.2340, 1.1116, 1.0915, 1.0442, 1.0378, 1.0299, 1.0350, 1.0736, 1.0914, 1.1154, 1.1455, 1.1797]
| block_min_k | 32 | 48 | 64 | 96 | 128 | 192 | 256 | 384 | 512 | 768 | 1024 | none |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| geomean | 1.2340 | 1.1116 | 1.0915 | 1.0442 | 1.0378 | 1.0299 | 1.0350 | 1.0736 | 1.0914 | 1.1154 | 1.1455 | 1.1797 |
| worst | 6.14x | 3.39x | 2.66x | 1.80x | 1.73x | 1.73x | 1.73x | 2.68x | 5.07x | 8.19x | 15.83x | 24.39x |
xychart-beta
title "Regret by block_min_k_batched"
x-axis "block_min_k_batched" [32, 48, 64, 96, 128]
y-axis "geometric-mean regret" 1.0 --> 1.08
line [1.0474, 1.0351, 1.0299, 1.0527, 1.0622]
| block_min_k_batched | 32 | 48 | 64 | 96 | 128 |
|---|---|---|---|---|---|
| geomean | 1.0474 | 1.0351 | 1.0299 | 1.0527 | 1.0622 |
| worst | 2.62x | 1.91x | 1.73x | 2.09x | 2.20x |
xychart-beta
title "Regret by block_min_batch"
x-axis "block_min_batch" [4, 16, 64, 256, 1024, 4096]
y-axis "geometric-mean regret" 1.0 --> 1.10
line [1.0692, 1.0492, 1.0299, 1.0406, 1.0625, 1.0882]
| block_min_batch | 4 | 16 | 64 | 256 | 1024 | 4096 |
|---|---|---|---|---|---|---|
| geomean | 1.0692 | 1.0492 | 1.0299 | 1.0406 | 1.0625 | 1.0882 |
| worst | 2.63x | 2.51x | 1.73x | 2.32x | 2.47x | 2.48x |
xychart-beta
title "Regret by qr_min_rows"
x-axis "qr_min_rows" [16, 32, 64, 128, 256, 512, 1024, 2048, none]
y-axis "geometric-mean regret" 1.0 --> 1.18
line [1.0634, 1.0634, 1.0634, 1.0545, 1.0393, 1.0299, 1.0467, 1.0972, 1.1672]
| qr_min_rows | 16 | 32 | 64 | 128 | 256 | 512 | 1024 | 2048 | none |
|---|---|---|---|---|---|---|---|---|---|
| geomean | 1.0634 | 1.0634 | 1.0634 | 1.0545 | 1.0393 | 1.0299 | 1.0467 | 1.0972 | 1.1672 |
| worst | 2.40x | 2.40x | 2.40x | 2.40x | 2.40x | 1.73x | 2.04x | 4.21x | 6.32x |
xychart-beta
title "Regret by qr_min_k"
x-axis "qr_min_k" [8, 16, 32, 64, 128, 256]
y-axis "geometric-mean regret" 1.0 --> 1.15
line [1.0471, 1.0471, 1.0299, 1.0416, 1.0685, 1.1392]
| qr_min_k | 8 | 16 | 32 | 64 | 128 | 256 |
|---|---|---|---|---|---|---|
| geomean | 1.0471 | 1.0471 | 1.0299 | 1.0416 | 1.0685 | 1.1392 |
| worst | 3.41x | 3.41x | 1.73x | 3.56x | 4.47x | 6.32x |
Held-out check of a batch-dependent block crossover (block from a smaller k once the batch is large enough), fitted on 151 points and scored on the other 115; the verdict is a bootstrap over the test points.
| rule | fitted on train | train geomean | test geomean | test worst | verdict |
|---|---|---|---|---|---|
| one block crossover | [“512”, “32”, “96”, “none”, “none”] | 1.0726 | 1.0605 | 2.00x | baseline |
| batch-dependent block crossover | {“block_min”: “192”, “block_lo”: “64”, “batch_hi”: “64”} | 1.0340 | 1.0247 | 1.50x | justified (better in 100% of resamples, median gain 3.4%) |
Best GPU backend per point (J whole-matrix kernel, B block kernel, j and b the same after QR), then what the rule picks:
M x N \ batch 1 4 16 64 256 1024 4096
4 x 4 J J J J J J J
8 x 8 J J J J J J J
16 x 8 J J J J J J J
32 x 8 J J J J J J J
64 x 8 J J J J J J J
128 x 8 J J J J J J J
256 x 8 J j J J J J J
16 x 16 J J J J J J J
32 x 16 j J J J J J J
64 x 16 J j J J J J J
128 x 16 J J J J J J J
256 x 16 j J J J J J J
512 x 16 j J J J J J J
32 x 32 J J J J J B B
64 x 32 j J J J J B B
128 x 32 J J J J J B B
256 x 32 J J J J B B B
512 x 32 J j J J B B B
1024 x 32 j j J j B B B
48 x 48 J J J J J J J
64 x 64 J J J J B B B
128 x 64 J J J J B B B
256 x 64 j J J B B B B
512 x 64 j j J B B B B
1024 x 64 j j j j B B B
2048 x 64 j j j j B B b
96 x 96 J J J B B B B
128 x 128 J J J B B B B
256 x 128 j j J B b b B
512 x 128 j j j B b b b
1024 x 128 j j j b b b b
2048 x 128 j j j b b b .
192 x 192 B B B B B B .
256 x 256 B B B B B B .
512 x 256 b B b b b b .
1024 x 256 b b b b b . .
2048 x 256 b b b b b . .
384 x 384 B B B B B . .
512 x 512 B B B B . . .
768 x 768 B B B . . . .
1024 x 1024 B B B . . . .
M x N \ batch 1 4 16 64 256 1024 4096
4 x 4 J J J J J J J
8 x 8 J J J J J J J
16 x 8 J J J J J J J
32 x 8 J J J J J J J
64 x 8 J J J J J J J
128 x 8 J J J J J J J
256 x 8 J J J J J J J
16 x 16 J J J J J J J
32 x 16 J J J J J J J
64 x 16 J J J J J J J
128 x 16 J J J J J J J
256 x 16 J J J J J J J
512 x 16 J J J J J J J
32 x 32 J J J J J J J
64 x 32 J J J J J J J
128 x 32 J J J J J J J
256 x 32 J J J J J J J
512 x 32 j j j j j j j
1024 x 32 j j j j j j j
48 x 48 J J J J J J J
64 x 64 J J J B B B B
128 x 64 J J J B B B B
256 x 64 J J J B B B B
512 x 64 j j j b b b b
1024 x 64 j j j b b b b
2048 x 64 j j j b b b b
96 x 96 J J J B B B B
128 x 128 J J J B B B B
256 x 128 J J J B B B B
512 x 128 j j j b b b b
1024 x 128 j j j b b b b
2048 x 128 j j j b b b .
192 x 192 B B B B B B .
256 x 256 B B B B B B .
512 x 256 b b b b b b .
1024 x 256 b b b b b . .
2048 x 256 b b b b b . .
384 x 384 B B B B B . .
512 x 512 B B B B . . .
768 x 768 B B B . . . .
1024 x 1024 B B B . . . .
GPU iff k <= gpu_max_k, batch * k >= gpu_min_batch_times_k and batch >= gpu_min_batch, with k = min(M, N), scored against the best of the CPU and the four Jacobi backends. worst is over the points where the chosen backend was timed; a pick the cost model had to guess is listed in the warnings instead.
| rule | geomean regret | worst | >10% | total time / oracle | est. picks |
|---|---|---|---|---|---|
| oracle (best per point) | 1.0000 | 1.00x | 0 | 1.000 | 0 |
| policy in effect (‘1024’, ‘256’, ‘4’) | 1.6904 | 6.70x | 177 | 2.547 | 0 |
| fitted (‘8’, ‘4096’, ‘1’) | 1.0019 | 1.25x | 3 | 1.000 | 0 |
9 of 462 combinations are within 0.5% of the best geomean: gpu_max_k 8 .. 8, gpu_min_batch_times_k 4096 .. 16384, gpu_min_batch 1 .. 16.
xychart-beta
title "Regret by gpu_min_batch_times_k"
x-axis "gpu_min_batch_times_k" [0, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, none]
y-axis "geometric-mean regret" 1.0 --> 1.21
line [1.1974, 1.0523, 1.0452, 1.0214, 1.0207, 1.0103, 1.0086, 1.0019, 1.0023, 1.0047, 1.0113]
| gpu_min_batch_times_k | 0 | 64 | 128 | 256 | 512 | 1024 | 2048 | 4096 | 8192 | 16384 | none |
|---|---|---|---|---|---|---|---|---|---|---|---|
| geomean | 1.1974 | 1.0523 | 1.0452 | 1.0214 | 1.0207 | 1.0103 | 1.0086 | 1.0019 | 1.0023 | 1.0047 | 1.0113 |
| worst | 46.81x | 6.04x | 4.49x | 2.59x | 2.59x | 2.17x | 2.17x | 1.25x | 1.25x | 1.25x | 1.68x |
xychart-beta
title "Regret by gpu_min_batch"
x-axis "gpu_min_batch" [1, 4, 16]
y-axis "geometric-mean regret" 1.0 --> 1.02
line [1.0019, 1.0019, 1.0019]
| gpu_min_batch | 1 | 4 | 16 |
|---|---|---|---|
| geomean | 1.0019 | 1.0019 | 1.0019 |
| worst | 1.25x | 1.25x | 1.25x |
xychart-beta
title "Regret by gpu_max_k"
x-axis "gpu_max_k" [8, 16, 32, 48, 64, 96, 128, 192, 256, 384, 512, 768, 1024, none]
y-axis "geometric-mean regret" 1.0 --> 1.39
line [1.0019, 1.0171, 1.0649, 1.0753, 1.1447, 1.1591, 1.2110, 1.2300, 1.3033, 1.3296, 1.3475, 1.3570, 1.3702, 1.3702]
| gpu_max_k | 8 | 16 | 32 | 48 | 64 | 96 | 128 | 192 | 256 | 384 | 512 | 768 | 1024 | none |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| geomean | 1.0019 | 1.0171 | 1.0649 | 1.0753 | 1.1447 | 1.1591 | 1.2110 | 1.2300 | 1.3033 | 1.3296 | 1.3475 | 1.3570 | 1.3702 | 1.3702 |
| worst | 1.25x | 2.26x | 2.80x | 2.80x | 3.63x | 3.63x | 3.63x | 4.39x | 4.84x | 6.70x | 6.70x | 6.70x | 6.70x | 6.70x |
Held-out check of a per-k boundary against the product rule, fitted on 151 points and scored on the other 115; the verdict is a bootstrap.
| rule | fitted on train | train geomean | test geomean | test worst | verdict |
|---|---|---|---|---|---|
| product rule | [8, 4096, 1] | 1.0026 | 1.0010 | 1.12x | baseline |
| per-k table | {“min_batch_by_k”: {“4”: 1024, “8”: 1024, “16”: 1024, “32”: null, “48”: null, “64”: null, “96”: null, “128”: null, “192”: null, “256”: null, “384”: null, “512”: null, “768”: null, “1024”: null}} | 1.0028 | 1.0204 | 2.26x | rejected (better in 0% of resamples, median gain -1.8%) |
Best backend per point (c CPU, J whole-matrix kernel, B block kernel, j and b the same after QR, . not measured), what the rule picks, and the speedup of the best GPU backend over the CPU:
M x N \ batch 1 4 16 64 256 1024 4096
4 x 4 c c c c c J J
8 x 8 c c c c c c J
16 x 8 c c c c c J J
32 x 8 c c c c c J J
64 x 8 c c c c c J J
128 x 8 c c c c c J J
256 x 8 c c c c c J J
16 x 16 c c c c c c c
32 x 16 c c c c c J J
64 x 16 c c c c c c J
128 x 16 c c c c c J c
256 x 16 c c c c c c c
512 x 16 c c c c c c c
32 x 32 c c c c c c c
64 x 32 c c c c c c c
128 x 32 c c c c c c c
256 x 32 c c c c c c c
512 x 32 c c c c c c c
1024 x 32 c c c c c c c
48 x 48 c c c c c c c
64 x 64 c c c c c c c
128 x 64 c c c c c c c
256 x 64 c c c c c c c
512 x 64 c c c c c c c
1024 x 64 c c c c c c c
2048 x 64 c c c c c c c
96 x 96 c c c c c c c
128 x 128 c c c c c c c
256 x 128 c c c c c c c
512 x 128 c c c c c c c
1024 x 128 c c c c c c c
2048 x 128 c c c c c c .
192 x 192 c c c c c c .
256 x 256 c c c c c c .
512 x 256 c c c c c c .
1024 x 256 c c c c c . .
2048 x 256 c c c c c . .
384 x 384 c c c c c . .
512 x 512 c c c c . . .
768 x 768 c c c . . . .
1024 x 1024 c c c . . . .
M x N \ batch 1 4 16 64 256 1024 4096
4 x 4 c c c c c J J
8 x 8 c c c c c J J
16 x 8 c c c c c J J
32 x 8 c c c c c J J
64 x 8 c c c c c J J
128 x 8 c c c c c J J
256 x 8 c c c c c J J
16 x 16 c c c c c c c
32 x 16 c c c c c c c
64 x 16 c c c c c c c
128 x 16 c c c c c c c
256 x 16 c c c c c c c
512 x 16 c c c c c c c
32 x 32 c c c c c c c
64 x 32 c c c c c c c
128 x 32 c c c c c c c
256 x 32 c c c c c c c
512 x 32 c c c c c c c
1024 x 32 c c c c c c c
48 x 48 c c c c c c c
64 x 64 c c c c c c c
128 x 64 c c c c c c c
256 x 64 c c c c c c c
512 x 64 c c c c c c c
1024 x 64 c c c c c c c
2048 x 64 c c c c c c c
96 x 96 c c c c c c c
128 x 128 c c c c c c c
256 x 128 c c c c c c c
512 x 128 c c c c c c c
1024 x 128 c c c c c c c
2048 x 128 c c c c c c .
192 x 192 c c c c c c .
256 x 256 c c c c c c .
512 x 256 c c c c c c .
1024 x 256 c c c c c . .
2048 x 256 c c c c c . .
384 x 384 c c c c c . .
512 x 512 c c c c . . .
768 x 768 c c c . . . .
1024 x 1024 c c c . . . .
M x N \ batch 1 4 16 64 256 1024 4096
4 x 4 0.02 0.10 0.17 0.82 0.64 1.10 1.68
8 x 8 0.04 0.13 0.40 0.58 0.73 0.89 1.01
16 x 8 0.05 0.13 0.35 0.39 0.88 1.16 1.33
32 x 8 0.06 0.15 0.37 0.66 0.67 1.13 1.39
64 x 8 0.06 0.17 0.43 0.71 0.98 1.09 1.42
128 x 8 0.07 0.21 0.22 0.79 0.46 1.23 1.22
256 x 8 0.09 0.12 0.45 0.79 0.88 1.22 1.05
16 x 16 0.09 0.15 0.38 0.56 0.63 0.68 0.76
32 x 16 0.07 0.12 0.48 0.42 0.92 1.04 1.25
64 x 16 0.15 0.13 0.34 0.76 0.77 0.98 1.12
128 x 16 0.15 0.25 0.58 0.76 0.73 1.02 0.96
256 x 16 0.12 0.26 0.45 0.70 0.74 0.88 0.81
512 x 16 0.16 0.31 0.63 0.77 0.70 0.47 0.44
32 x 32 0.19 0.26 0.50 0.46 0.51 0.58 0.69
64 x 32 0.13 0.28 0.58 0.53 0.64 0.81 0.90
128 x 32 0.22 0.31 0.59 0.56 0.66 0.71 0.78
256 x 32 0.25 0.33 0.63 0.54 0.58 0.60 0.64
512 x 32 0.26 0.34 0.49 0.49 0.47 0.57 0.55
1024 x 32 0.32 0.45 0.54 0.44 0.47 0.58 0.53
48 x 48 0.15 0.20 0.35 0.37 0.43 0.42 0.42
64 x 64 0.19 0.22 0.34 0.32 0.42 0.50 0.48
128 x 64 0.24 0.28 0.43 0.46 0.62 0.64 0.60
256 x 64 0.23 0.31 0.42 0.44 0.59 0.59 0.57
512 x 64 0.31 0.41 0.47 0.52 0.57 0.59 0.54
1024 x 64 0.38 0.52 0.50 0.54 0.57 0.56 0.56
2048 x 64 0.48 0.57 0.57 0.53 0.59 0.58 0.47
96 x 96 0.24 0.29 0.33 0.36 0.49 0.45 0.44
128 x 128 0.21 0.27 0.34 0.42 0.45 0.41 0.40
256 x 128 0.28 0.34 0.40 0.59 0.58 0.55 0.53
512 x 128 0.33 0.33 0.50 0.56 0.58 0.54 0.52
1024 x 128 0.39 0.38 0.58 0.60 0.63 0.60 0.51
2048 x 128 0.46 0.42 0.61 0.64 0.67 0.64 .
192 x 192 0.21 0.21 0.25 0.28 0.25 0.23 .
256 x 256 0.30 0.30 0.26 0.24 0.22 0.21 .
512 x 256 0.44 0.43 0.35 0.34 0.32 0.28 .
1024 x 256 0.48 0.43 0.40 0.40 0.38 . .
2048 x 256 0.57 0.47 0.50 0.48 0.46 . .
384 x 384 0.34 0.30 0.20 0.16 0.15 . .
512 x 512 0.52 0.38 0.18 0.16 . . .
768 x 768 0.52 0.33 0.15 . . . .
1024 x 1024 0.67 0.31 0.25 . . . .
Where the rule above chooses the CPU, the bidiag backend (GPU bidiagonalization, then LAPACK’s bidiagonal solve) from a threshold k = min(M, N) on (0: never), for batches up to a cap (0: any; it solves a batch one matrix after another, the CPU path spreads one over every core), fitted on the points where bidiag was timed (k >= 128, within the cost cap): the region the threshold decides. With vectors it is scored against the best of all backends, bidiag included; for singular values alone (svdvals), bidiag against the CPU at the points where the rule chooses the CPU.
| threshold | batch cap | geomean regret | worst | without bidiag: geomean | worst | held out (fitted on half) | |
|---|---|---|---|---|---|---|---|
| with vectors | 1024 | 1 | 1.0000 | 1.00x | 1.0273 | 2.03x | from 1536, batch <= 1: 1.0007 vs 1.0136 |
| singular values alone | 2048 | 1 | 1.0000 | 1.00x | 1.0203 | 1.91x | from 3072, batch <= 1: 1.0056 vs 1.0056 |
bidiag over the CPU (with vectors), M x N x batch: 128x128x1 0.24x, 128x128x4 0.08x, 128x128x16 0.02x, 128x128x64 0.02x, 128x128x256 0.02x, 256x128x1 0.28x, 256x128x4 0.10x, 256x128x16 0.03x, 256x128x64 0.03x, 256x128x256 0.03x, 512x128x1 0.33x, 512x128x4 0.11x, 512x128x16 0.05x, 512x128x64 0.04x, 512x128x256 0.03x, 1024x128x1 0.39x, 1024x128x4 0.13x, 1024x128x16 0.06x, 1024x128x64 0.05x, 1024x128x256 0.05x, 2048x128x1 0.44x, 2048x128x4 0.14x, 2048x128x16 0.08x, 2048x128x64 0.07x, 2048x128x256 0.06x, 192x192x1 0.27x, 192x192x4 0.07x, 192x192x16 0.03x, 192x192x64 0.02x, 192x192x256 0.02x, 256x256x1 0.39x, 256x256x4 0.10x, 256x256x16 0.04x, 256x256x64 0.03x, 512x256x1 0.44x, 512x256x4 0.12x, 512x256x16 0.05x, 512x256x64 0.04x, 1024x256x1 0.49x, 1024x256x4 0.14x, 1024x256x16 0.06x, 1024x256x64 0.05x, 2048x256x1 0.56x, 2048x256x4 0.17x, 2048x256x16 0.08x, 2048x256x64 0.07x, 384x384x1 0.45x, 384x384x4 0.13x, 384x384x16 0.05x, 384x384x64 0.04x, 512x512x1 0.65x, 512x512x4 0.18x, 512x512x16 0.07x, 512x512x64 0.06x, 768x768x1 0.73x, 768x768x4 0.22x, 768x768x16 0.11x, 1024x1024x1 1.02x, 1024x1024x4 0.30x, 1024x1024x16 0.25x, 1536x1536x1 1.12x, 1536x1536x2 0.64x, 1536x1536x4 0.35x, 2048x2048x1 1.43x, 2048x2048x2 0.92x, 2048x2048x4 0.66x, 3072x3072x1 1.89x, 4096x4096x1 2.03x
bidiag over the CPU (singular values alone), M x N x batch: 128x128x1 0.18x, 128x128x4 0.07x, 128x128x16 0.02x, 128x128x64 0.02x, 128x128x256 0.02x, 256x128x1 0.21x, 256x128x4 0.07x, 256x128x16 0.02x, 256x128x64 0.02x, 256x128x256 0.02x, 512x128x1 0.22x, 512x128x4 0.07x, 512x128x16 0.03x, 512x128x64 0.02x, 512x128x256 0.02x, 1024x128x1 0.25x, 1024x128x4 0.07x, 1024x128x16 0.04x, 1024x128x64 0.03x, 1024x128x256 0.02x, 2048x128x1 0.27x, 2048x128x4 0.10x, 2048x128x16 0.05x, 2048x128x64 0.03x, 2048x128x256 0.03x, 192x192x1 0.15x, 192x192x4 0.05x, 192x192x16 0.02x, 192x192x64 0.01x, 192x192x256 0.01x, 256x256x1 0.21x, 256x256x4 0.06x, 256x256x16 0.03x, 256x256x64 0.02x, 512x256x1 0.24x, 512x256x4 0.07x, 512x256x16 0.03x, 512x256x64 0.02x, 1024x256x1 0.27x, 1024x256x4 0.09x, 1024x256x16 0.03x, 1024x256x64 0.03x, 2048x256x1 0.31x, 2048x256x4 0.10x, 2048x256x16 0.05x, 2048x256x64 0.04x, 384x384x1 0.28x, 384x384x4 0.08x, 384x384x16 0.04x, 384x384x64 0.03x, 512x512x1 0.41x, 512x512x4 0.12x, 512x512x16 0.05x, 512x512x64 0.04x, 768x768x1 0.51x, 768x768x4 0.14x, 768x768x16 0.10x, 1024x1024x1 0.74x, 1024x1024x4 0.22x, 1024x1024x16 0.29x, 1536x1536x1 0.89x, 1536x1536x2 0.53x, 1536x1536x4 0.33x, 2048x2048x1 1.17x, 2048x2048x2 0.82x, 2048x2048x4 0.68x, 3072x3072x1 1.75x, 4096x4096x1 1.91x
Pass-to-pass ratio, 1200 measurements: median 1.010, p90 1.063, max 2.75.
| runtime | n | median | p90 | max |
|---|---|---|---|---|
| <1 ms | 261 | 1.035 | 1.387 | 2.75 |
| 1-3 ms | 172 | 1.010 | 1.051 | 1.28 |
| 3-10 ms | 208 | 1.006 | 1.025 | 1.15 |
| 10-30 ms | 152 | 1.010 | 1.023 | 1.07 |
| 30-100 ms | 150 | 1.005 | 1.024 | 1.12 |
| >100 ms | 257 | 1.007 | 1.032 | 1.13 |