High warm start delay

[leokondrashov]

Version

main branch

Steps to reproduce

Run RPS mode with all warm start (RpsColdStartRatioPercentage is 0) and RPS of 1.

Expected behavior

Fast warmups after stabilization period (~5ms)

Observed behavior

Big delay, ~50ms

Additional notes

It's been reported that this problem diminishes with higher RPS.
A possible cause is the switch to new gRPC invocation semantics using client instead of connection establishment.

[leokondrashov]

With a higher invocation rate (e.g., 10RPS), the problem diminishes but doesn't go away completely. It still affects ~10% of invocations.