Golang net/http: reuse gzip reader from pool for transport

https://github.com/golang/go/blob/089e37a931d30d4055c7468facb602c4cfa9b537/src/net/http/transport.go#L2858 Is it possible to reuse this gzip reader from a sync.Pool to reduce the memory allocation cause by flate.NewReader

Comment From: bcmills

gzip.Reader does have a Reset method, but we would also need some way to know when the gzip.Reader is safe to reuse.

Hmm. (*http.gzipReader).Read already has some locking in Read. Perhaps we could do something like:

func (gz *gzipReader) Read(p []byte) (n int, err error) {
    gz.body.mu.Lock()
    if gz.body.closed {
        err = errReadOnClosedResBody
    } else if gz.zr == nil {
        if gz.zerr == nil {
            zr := gzipPool.Get().(*gzip.Reader)
            gz.zerr = zr.Reset(gz.body)
            if gz.zerr == nil {
                gz.zr = zr
            } else {
                gzipPool.Put(zr)
            }
        }
        err = gz.zerr
    }
    gz.body.mu.Unlock()

    if err != nil {
        return 0, err
    }
    return gz.zr.Read(p)
}

func (gz *gzipReader) Close() error {
    err := gz.body.Close()

    gz.body.mu.Lock()
    if gz.zr != nil {
        gz.zr.Reset(emptyReader{})  // drop reference to gz.body
        gzipPool.Put(gz.zr)
        gz.zr = nil
    }
    gz.body.mu.Unlock()

    return err
}

var (
    gzipPool = sync.Pool{
        New: func() any { return new(gzip.Reader) },
    }
    _ flate.Reader = emptyReader{}
)

type emptyReader struct{}

func (emptyReader) Read([]byte) (int, error) { return 0, io.EOF }
func (emptyReader) ReadByte() (byte, error) { return 0, io.EOF }

Of course, such a change would also need a supporting benchmark. @Sunyue, want to send it for Go 1.22?

(CC @neild)

Comment From: neild

Seems like something we could do; I'm happy to review a CL if someone wants to send me one.

Comment From: gopherbot

Change https://go.dev/cl/510115 mentions this issue: net/http: pool transport gzip readers

Comment From: gopherbot

Change https://go.dev/cl/510255 mentions this issue: net/http: pool transport gzip readers

Comment From: AlexanderYastrebov

Created https://github.com/golang/go/pull/61390

Comment From: unclejack

Is there any technical blocker for these changes? The use of a pool for the gzip.Reader will help avoid memory allocations.