Proposal Details

Summery

Getting the underlying socket file descriptor is required when sending eg. shutdown signals to the socket, while keeping the http server running.

Why

By having a listener with SO_REUSEPORT enabled, there is no good way to tell the kernel to stop queuing requests for the listening service. The only I found to stop queuing requests is to shutdown the socket, but to do that we need the file descriptor for the socket.

By calling listener.Close() or http.Shutdown() will not tell the kernel to stop queuing requests to the service. If this is called while the kernel is queuing requests, all requests in queue (and new requests) will fail until the service stops. This would result in a lot of failed requests.

Example

Code 
package main

import (
    "context"
    "fmt"
    "log"
    "net"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"

    "golang.org/x/sys/unix"
)

func main() {
    mux := http.NewServeMux()

    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello world\n")
    })

    svc := http.Server{
        Handler: mux,
        Addr:    ":8000",
    }

    var socketFd uintptr

    listener := net.ListenConfig{}

    listener.Control = func(_, _ string, c syscall.RawConn) error {
        var opErr error

        err := c.Control(func(fd uintptr) {
            // Not recomended to do this, because the file descriptor is not
            // guaranteed to remain valid after this function finishes.
            socketFd = fd
            opErr = unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1)
        })
        if err != nil {
            return err
        }

        return opErr
    }

    go func() {
        l, err := listener.Listen(context.Background(), "tcp", ":8000")
        if err != nil {
            panic(err)
        }

        log.Fatal(svc.Serve(l))
    }()

    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt)
    <-c

    log.Print("Shutting down service...")

    err := syscall.Shutdown(int(socketFd), syscall.SHUT_WR)
    if err != nil {
        panic(err)
    }

    // wait for the queue to drain
    time.Sleep(5 * time.Second)

    ctx, cancelFunc := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancelFunc()

    if err := svc.Shutdown(ctx); err != nil {
        log.Fatalln("Failed to shutdown http service:", err)
    }

    log.Print("Server stopped")
}

In this example, I manually call syscall.Shutdown(int(socketFd), syscall.SHUT_WR) to signal the shutdown signal to the socket.

In this code, we save the socket file descriptor during the Control function, which is not recommended, as the file descriptor is not guaranteed to be valid after the Control function. Ideally, we can use a method to get the underlying socket file descriptor, or have a method to tell the kernel to stop queuing requests for the socket.

Addinital details (as I understand it)

Enabling SO_REUSEPORT allows multiple services to listen on the same port, the kernel will use hash-based load balancing to determine where to route an incoming request. The request then sits in a queue for the specific service, then the service can accept the request when its ready.

Under heavy load, even by calling syscall.Shutdown() still occasional results in few failed requests, that might be because there is still a few requests in queue, or maybe not the right shutdown signal, I'm not completely sure. However, its still many times less failed requests then just shutdowning the listener or http server. Ideally this should have zero failed requests even under heavy load.

Comment From: gabyhelp

Related Issues and Documentation

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: ianlancetaylor

This idea only makes sense for a TCP listener. So you can already do something like (untested)

    c, err := l.(*net.TCPListener).SyscallConn()
    if err != nil { panic("listener already closed?") }
    err = c.Control(func(fd uintptr) {
        if err := syscall.Shutdown(int(fd)); err != nil {
            log.Error(err)
        }
    })

Comment From: WestleyK

Thank you @ianlancetaylor for that snippet, that seem to mostly work, I'm still getting some failed requests reported by ab under light load, however its overall very similar to my code.

I do wish there was a more reliable way to "shutdown" the socket to stop the kernel from queuing requests, but my original question has been answered by your snippet, thank you very much.

Comment From: WestleyK

Update: My test code was faulty, the svc.Serve(l) should not exit the main thread on serve error (at least not stopped error, need to add error check in real code). In addition, calling syscall.Close(int(fd)) is sufficient.