ci: Cache hyperfine results from main

[larseggert]

So we can in a second PR compute deltas for PRs and display them.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.28%. Comparing base (b9071e4) to head (84462cb).
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2417   +/-   ##
=======================================
  Coverage   95.28%   95.28%           
=======================================
  Files         114      114           
  Lines       37111    37111           
  Branches    37111    37111           
=======================================
  Hits        35363    35363           
  Misses       1742     1742           
  Partials        6        6           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to b9071e4.

neqo-latest as client

• neqo-latest vs. kwik: run cancelled after 20 min
• neqo-latest vs. xquic: run cancelled after 20 min

neqo-latest as server

• ngtcp2 vs. neqo-latest: run cancelled after 20 min

All results

Succeeded Interop Tests

None ❓

Unsupported Interop Tests

None ❓

[github-actions]

Benchmark results

decode 4096 bytes, mask ff:

       time:   [11.797 µs 11.888 µs 12.047 µs]
Found 16 outliers among 100 measurements (16.00%)
  1 (1.00%) low severe
  4 (4.00%) low mild
  11 (11.00%) high severe

decode 1048576 bytes, mask ff:

       time:   [2.9014 ms 2.9108 ms 2.9217 ms]
Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) high severe

decode 4096 bytes, mask 7f:

       time:   [19.690 µs 19.737 µs 19.792 µs]
Found 16 outliers among 100 measurements (16.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  1 (1.00%) high mild
  12 (12.00%) high severe

decode 1048576 bytes, mask 7f:

       time:   [4.7148 ms 4.7261 ms 4.7390 ms]
Found 14 outliers among 100 measurements (14.00%)
  1 (1.00%) low mild
  13 (13.00%) high severe

decode 4096 bytes, mask 3f:

       time:   [6.2226 µs 6.2534 µs 6.2910 µs]
Found 15 outliers among 100 measurements (15.00%)
  5 (5.00%) low mild
  3 (3.00%) high mild
  7 (7.00%) high severe

decode 1048576 bytes, mask 3f:

       time:   [2.1122 ms 2.1190 ms 2.1260 ms]
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high severe

coalesce_acked_from_zero 1+1 entries:

       time:   [93.115 ns 93.420 ns 93.759 ns]
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe

coalesce_acked_from_zero 3+1 entries:

       time:   [110.72 ns 111.06 ns 111.42 ns]
Found 19 outliers among 100 measurements (19.00%)
  3 (3.00%) low mild
  4 (4.00%) high mild
  12 (12.00%) high severe

coalesce_acked_from_zero 10+1 entries:

       time:   [110.33 ns 110.78 ns 111.31 ns]
Found 13 outliers among 100 measurements (13.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  1 (1.00%) high mild
  7 (7.00%) high severe

coalesce_acked_from_zero 1000+1 entries:

       time:   [92.904 ns 98.266 ns 110.09 ns]
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe

RxStreamOrderer::inbound_frame():

       time:   [111.78 ms 111.90 ms 112.12 ms]
Found 16 outliers among 100 measurements (16.00%)
  7 (7.00%) low mild
  8 (8.00%) high mild
  1 (1.00%) high severe

SentPackets::take_ranges:

       time:   [5.2454 µs 5.4257 µs 5.6219 µs]
Found 8 outliers among 100 measurements (8.00%)
  7 (7.00%) high mild
  1 (1.00%) high severe

transfer/pacing-false/varying-seeds:

       time:   [37.220 ms 37.306 ms 37.402 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

transfer/pacing-true/varying-seeds:

       time:   [37.199 ms 37.281 ms 37.370 ms]
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

transfer/pacing-false/same-seed:

       time:   [37.332 ms 37.408 ms 37.491 ms]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

transfer/pacing-true/same-seed:

       time:   [37.766 ms 37.836 ms 37.915 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client:

       time:   [838.86 ms 850.51 ms 862.63 ms]
       thrpt:  [115.92 MiB/s 117.58 MiB/s 119.21 MiB/s]

1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client:

       time:   [317.04 ms 320.69 ms 324.44 ms]
       thrpt:  [30.822 Kelem/s 31.182 Kelem/s 31.542 Kelem/s]

1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client:

       time:   [25.461 ms 25.592 ms 25.722 ms]
       thrpt:  [38.878  elem/s 39.075  elem/s 39.276  elem/s]

1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client:

       time:   [1.8225 s 1.8409 s 1.8590 s]
       thrpt:  [53.791 MiB/s 54.322 MiB/s 54.869 MiB/s]

Client/server transfer results

Transfer of 33554432 bytes over loopback, 20 runs.

Client	Server	CC	Pace	Mean [ms]	Min [ms]	Max [ms]
neqo	neqo	reno	on	452.1 ± 17.9	426.5	491.9
neqo	neqo	reno		435.3 ± 21.7	414.0	508.2
neqo	neqo	cubic	on	435.9 ± 11.1	419.4	469.5
neqo	neqo	cubic		445.4 ± 14.8	423.7	485.9
google	neqo	reno	on	824.1 ± 144.4	574.8	1093.6
google	neqo	reno		827.1 ± 133.0	580.4	984.6
google	neqo	cubic	on	827.5 ± 145.1	592.2	1122.1
google	neqo	cubic		814.4 ± 131.2	576.5	965.9
google	google			525.7 ± 15.6	509.3	581.5
neqo	msquic	reno	on	237.2 ± 49.8	208.0	422.3
neqo	msquic	reno		219.3 ± 13.5	202.3	247.7
neqo	msquic	cubic	on	221.1 ± 18.2	200.8	287.8
neqo	msquic	cubic		223.7 ± 30.1	200.6	346.6
msquic	msquic			121.5 ± 47.5	101.5	321.2

⬇️ Download logs

[mxinden]

🚀 thanks!