Change default chunk payload string length #1039
Conversation
api/auth/signer/v4/chunked_reader.go
Outdated
| @@ -37,7 +40,7 @@ var ( | |||
| // The chunkedReader returns io.EOF when the final 0-length chunk is read. | |||
| func NewChunkedReader(r io.ReadCloser, streamSigner *ChunkSigner) io.ReadCloser { | |||
| return &chunkedReader{ | |||
| r: bufio.NewReader(r), | |||
| r: bufio.NewReaderSize(r, MaxChunkedPayloadLineLength), | |||
There was a problem hiding this comment.
I don't think it's a fix. I can always create something that is larger than this limit and make the gateway panic which is unacceptable.
There was a problem hiding this comment.
I'm not sure too, but we would try
api/auth/signer/v4/chunked_reader.go
Outdated
| @@ -199,7 +202,7 @@ func readChunkLine(b *bufio.Reader) ([]byte, []byte, error) { | |||
| } | |||
| return nil, nil, err | |||
| } | |||
| if len(p) >= maxLineLength { | |||
| if len(p) >= MaxChunkedPayloadLineLength { | |||
There was a problem hiding this comment.
And here it's too late, panic happens earlier than that. The real question is why ReadSlice doesn't return ErrBufferFull above.
There was a problem hiding this comment.
Moving these rows upper is not a solution either, because panic inside bufio.ReadSlice and we just don't have the opportunity to check the length
There was a problem hiding this comment.
I know, but we also need to know why it panics. It shouldn't.
|
Can be related to multithreading in fact. Why does it happen in a separate routine? You can use https://pkg.go.dev/io#TeeReader instead of |
Signed-off-by: Evgenii Baidakov <evgenii@nspcc.io>
smallhive
force-pushed
the
1032-panic-under-high-load
branch
from
327cf91 to
19fb4cc
Compare
I don't think this fix will change the situation, but it looks better. Maybe this panic is connected to golang/go#22330 |
Closes #1032.
We should try it out