I recently spent some time figuring out how to gracefully shut down a Go service. The goal was to allow in-flight transactions to complete successfully before shutting down, but return Unavailable for any new requests. I found the solution to be fairly straightforward for Linux, but a little bit more tricky for Windows, specifically when running in a Windows Docker container.

General solution for handling termination signals (Linux/Darwin) #

For most use cases, the os/signal package works for capturing termination signals:

stop := make(chan os.Signal)
signal.Notify(stop, syscall.SIGTERM, syscall.SIGINT)
 
go func() {
    defer signal.Stop(stop)
    sig := <-stop
    action()
}()

This solution works for Linux, Darwin (Mac), and even works for a regular Windows installation (not in Docker).

Windows specific solution for handling termination signals #

Unfortunately, while the above solution works for most use cases, it does not work when run inside a Windows Docker container. The only way to handle termination inside a Windows Docker container is to use the Console API SetConsoleCtrlHandler function:

kernel32 := syscall.NewLazyDLL("kernel32.dll")
setConsoleCtrlHandler := kernel32.NewProc("SetConsoleCtrlHandler")
 
result, _, err := setConsoleCtrlHandler.Call(
    syscall.NewCallback(func(controlType uint) uint {
        action()
        return 0
    }), 1)

The available control types are described here: https://docs.microsoft.com/en-us/windows/console/handlerroutine#parameters

You need to be using one of the newer versions of the Windows containers and there are a few registry settings that need to be set in the Dockerfile:

FROM mcr.microsoft.com/windows/servercore:ltsc2019
USER ContainerAdministrator
RUN reg add hklm\system\currentcontrolset\services\cexecsvc /v ProcessShutdownTimeoutSeconds /t REG_DWORD /d 60
RUN reg add hklm\system\currentcontrolset\control /v WaitToKillServiceTimeout /t REG_SZ /d 60000 /f

This provides the same level of functionality on Windows as the previous solution does for Linux/Darwin.

How to shut down a gRPC server #

Now that we can handle termination signals from Linux/Darwin/Windows, we need to actually stop the service gracefully. The gRPC server method GracefulStop will allow any in-flight requests to complete and return status code Unavailable for any new requests. Use this method to gracefully shut down your gRPC server when you have received a termination signal:

// server is *grpc.Server
server.GracefulStop()

If you are using grpc-gateway to route HTTP requests to a gRPC server, make sure to stop the gRPC server first, before stopping the gateway.

How to shut down an HTTP server #

The HTTP server method Shutdown provides the same functionality as GracefulStop, but for an HTTP server. Use this method to gracefully shut down your HTTP server when you have received a termination signal:

// server is *http.Server
err := server.Shutdown(context.Background())
if err != nil {
    return err
}

The context can be used to provide a deadline for shutting down the HTTP server.

As a final note, it’s definitely a good idea to be on the latest patched version of whatever OS you are using. Aside from security concerns, some older Windows versions will not work with the above solution. Hopefully this post helps someone else trying to handle scaling down services without abruptly terminating in-flight requests.