Proposal

Add a new WithHandle method for os.Process, which, if a handle is available, calls a specified callback function with the process handle as an argument. On Linux, handle is a pidfd (file descriptor referring to the process). On Windows, handle is a handle to the process.

// WithHandle calls a supplied function f with the process handle as an argument.
// If a process handle is not available, the function is not called. The handle is
// guaranteed to be correct until f returns.
//
// Currently, process handle is only available for Windows and modern Linux
// systems (where it is known as pidfd).
func (p *Process) WithHandle(f func(handle uintptr))

Context

This is a continuation of #62654, in particular its item 6, described in there as:

  1. (Optional) Add (*Process).Handle() uintptr method to return process handle on Windows and pidfd on Linux. This might be useful for low-level operations that require handle/pidfd (e.g. pidfd_getfd on Linux), or to ensure that pidfd (rather than pid) is being used for kill/wait.

Also, a similar thing was proposed earlier here by @prattmic:

A new os.Process.Fd() could return the pid FD for additional direct use.

Use cases

1. Check if pidfd is being used on Linux.

Since Go 1.23, pidfd is used for os.Process-related operations instead of pid, if supported by the Linux kernel. This includes os.StartProcess and os.FindProcess (they obtain a pidfd), as well as (*Process).Wait, (*Process).Signal, and (*Process).Kill (they use pidfd). The main benefit of pidfd in the use cases above is a guarantee we're referring to the same process (i.e. there's no pid reuse issue).

However, since this is done in a fully transparent way, there is no way for a user to know if pidfd is being used or not. Some programs implement some protection against pid reuse (for example, runc and cri-o obtain and check process start time from /proc/<pid>/stat). They can benefit from being able to know if Go is using pidfd internally.

Another example is containerd which relies on Go 1.23 using pidfd internally, but since there's no way to check they had to recreate all the functionality checking for pidfd support here (which is still not 100% correct since the checks are slightly different from those in Go's checkPidfd, and Go checks may change over time ). Cc @fuweid.

With the proposed interface, a user can easily check if pidfd is being used:

p, err := os.FindProcess()
pidfdUsed := false
p.WithHandle(func(_ uintptr) {
       pidfdUsed = true
})

2. Use the existing pidfd directly.

Aside from use cases already covered by existing os.Process methods, pidfd can also be used to: - obtain a duplicate of a file descriptor of another process (pidfd_getfd(2)); - select/poll/epoll on a pidfd to know when a process is terminated; - move a process/thread into one or more of the same namespaces as the process referred to by the file descriptor (setns(2)).

Other use cases may emerge in the future.

Currently, the only way to obtain a pidfd on Linux is to execute a new process (via os.StartProcess or os/exec) with process' Attr.SysAttr.PidFD field set. This works if we're starting the process, but not in any other case (someone else starts a process for us, or it is already running).

Questions / discussion points

1. What are (could be) the additional direct use cases of Windows process handle?

A few are listed here. Apparently some git grep (GetPriorityClass, SetPriorityClass, AssignProcessToJobObject) are implemented in golang.org/x/sys/windows.

2. Should a duplicate of a handle be used, or the original handle?

Use the original one.

Arguments against duplicated handle: - a duplicated pidfd makes the "check if pidfd is being used" use case above more complicated as the user will need to close the returned pidfd; - a user can always use dupfd if/when needed; - returned handle won't leak as it is still part of os.Process struct, which have a Release method and a proper finalizer; - os.File.Fd returns the original underlying fd, pidfd is similar.

Arguments for duplicated handle: - cleaner separation of responsibilities(?); - a Windows process handle can be duplicated;

3. Should WithHandle provide *os.File rather than uintptr?

Raw handle makes more sense in this case, and *os.File won't work with Windows process handle as it is not a file descriptor.

4. Should this be Linux-specific?

Probably not. Since we have a boolean flag returned, we can implement it for all platforms and do nothing for those that do not support process handle (other than Windows and Linux).

Comment From: gabyhelp

Related Issues

Related Code Changes

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: fuweid

Thanks @kolyshkin for tagging me. This proposal looks good me from user perspective. I don't need to check pidfd in my side. :)

Comment From: adonovan

It should probably return an error instead of a boolean since the operation is (I think?) a system call and could in principle fail for more reasons than ENOTSUP.

Comment From: kolyshkin

It should probably return an error instead of a boolean since the operation is (I think?) a system call and could in principle fail for more reasons than ENOTSUP.

The pidfd is already used internally by os, and Handle does not do any syscalls, it merely returns an existing pidfd, if available. An error getting pidfd can happen way before the call to Handle.

Are you suggesting we need to save the error getting pidfd in struct Process, and have Handle return it?

I've added bool because 0 is a valid fd and the type is unsigned, so it's not trivial for the caller to distinguish between a valid pidfd and "pidfd is not available".

Comment From: adonovan

The pidfd is already used internally by os, and Handle does not do any syscalls, it merely returns an existing pidfd, if available.

I see. Thanks for clarifying.

Are you suggesting we need to save the error getting pidfd in struct Process, and have Handle return it?

No, what you have seems fine.

I've added bool because 0 is a valid fd and the type is unsigned, so it's not trivial for the caller to distinguish between a valid pidfd and "pidfd is not available".

That seems appropriate.

Comment From: kolyshkin

Not sure if there's anything I can do to move this forward. This seems to be the last step needed to fully support Linux pidfd in Go, and it also seems pretty trivial (except, unlike all the previous work, it amends the public API of os).

Any suggestions to move this forward @adonovan @prattmic @ianlancetaylor?

Comment From: adonovan

I don't think there's anything lacking in this proposal; the only obstacle is that the committee is not keeping up with the volume of proposals. We'll be scheduling extra sessions in the run up to the go1.25 freeze, and longer term we're thinking about ways to improve our efficiency. Thanks for your patience.

Comment From: prattmic

I think having an API for this makes sense.

That said, I am a bit concerned about returning the raw FD, similar to os.(*File).Fd.

One of the most common footguns users run into with finalizers (if I may be so bold, probably the most common) is that the fd returned from os.(*File).Fd does not keep the File alive. File has a finalizer that closes the FD, so if you don't keep the file alive (usually with defer f.Close() or explicit runtime.KeepAlive) then you can find the FD randomly closed out from under you.

Process also has a finalizer (well, cleanup as of 1.25) that closes the pidfd, so I believe this API will suffer from the exact same footgun. I'd like to avoid that footgun on a new API if we can.

Off the top of my head, a couple of ways to avoid this problem:

  1. Handle dups the FD before returning it. You discuss this above but don't mention that it avoids the finalizer issue, which is a significant advantage.

  2. Using some sort of callback API like func (*Process) WithHandle(f func(handle uintptr)). We have some existing use of this pattern with syscall.RawConn, which net uses to provide access to the underlying FD. e.g., net.(*IPConn).SyscallConn. This approach doesn't completely prevent misuse (you can keep a copy of the FD after the callback returns), but it is much safer by default.

Comment From: kolyshkin

The callback API idea sounds really appealing to me! Perhaps I should rewrite this proposal to use it (or write a new one? or will you @prattmic write it?)

Comment From: prattmic

You can update this proposal. No need to file a new one for every change.

Comment From: aclements

Our main concern is that this API would not prevent the runtime from closing this FD/handle. This is essentially why os.File originally had File.FD, but we later added File.SyscallConn.

Comment From: prattmic

@aclements Could you clarify? Is your concern essentially what I said in https://github.com/golang/go/issues/70352#issuecomment-2859873287, or is there a concern beyond that? Note that @kolyshkin states just above their intention to switch to a callback API.

Comment From: aclements

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — aclements for the proposal review group

Comment From: kolyshkin

I've updated the proposal to use a WithHandle callback, will self-review it tomorrow, in the meantime please let me know if there are any issues.

Comment From: aclements

Could you clarify? Is your concern essentially what I said in https://github.com/golang/go/issues/70352#issuecomment-2859873287,

Nope, you're right. My concern is exactly what you said above. @kolyshkin , thanks for updating the proposal to use a callback!

Comment From: aclements

WithHandle needs an error result to indicate when no handle is available (for other platforms, or if pidfd isn't available on Linux), but otherwise this API looks good.

Comment From: kolyshkin

WithHandle needs an error result to indicate when no handle is available (for other platforms, or if pidfd isn't available on Linux), but otherwise this API looks good.

My idea is, if handle is not available, the callback function is not called. The benefit being is it simplifies usage, the consequence is there's no way to know why pidfd/handle is not available.

Now, on Windows the handle is always available, so there's no need for an error. For Linux, the reason could be older kernel, or pidfd syscalls not being allowed by seccomp, or otherwise misbehaving (see checkPidfd). For all other OSes it's always not available (as of now).

So, we could be interested in an error only on Linux, and pidfd being unavailable is quite common (as many people run older kernels).

Finally, I can't think of a scenario in which a specific reason why pidfd is not available can be useful in the code (other than logging a specific reason, that is).

Comment From: aclements

Sorry, but I don't understand your argument. If the callback is doing anything interesting, presumably the caller needs some alternative if the handle isn't available. If we don't give easy access to that information, there's just going to be a pattern of "ok := false; p.WithHandle(func() { ok = true }); if !ok { fallback }", which seems terrible.

Comment From: aclements

I propose that the API should be:

// WithHandle calls a supplied function f with the process handle as an argument.
// If a process handle is not available, it returns ErrNoHandle. The handle is
// guaranteed to exist and refer to this process until f returns.
//
// Currently, process handles are only available on Linux 5.4 and later
// (where it is known as pidfd) and Windows.
func (p *Process) WithHandle(f func(handle uintptr)) error

var ErrNoHandle = errors.New("process handle unavailable")

I went back and forth on whether ErrNoHandle should wrap errors.ErrUnsupported, but I'm not sure if there are cases where process handles are supported by the environment, but we don't have one for a given process for whatever reason. And introducing two different errors for "unsupported" versus "we don't happen to have one for this process" seemed like a needless complication that could also introduce bugs where a caller handles one but not the other.

Comment From: adonovan

Wording tweak:

// WithHandle calls a supplied function f with a valid process handle as an argument.
// The handle is guaranteed to refer to process p until f returns, even if p terminates.
// If no process handle is available, it returns ErrNoHandle.
...

("guaranteed to exist" conflicts with "if not available")

Comment From: prattmic

I'm not sure if there are cases where process handles are supported by the environment, but we don't have one for a given process for whatever reason.

If nothing else, p := os.Process{Pid: 1234} will create such a Process. (Yes, we unfortunately have to support that because that form is in the wild). Personally I think ErrNoHandle is fine in this case.

Comment From: prattmic

(WithHandle could attempt to open a new pidfd in that case, but currently none of the other Process methods do that. The pidfd is only opened when the Process is created.)

Comment From: aclements

Based on the discussion above, this proposal seems like a likely accept. — aclements for the proposal review group

// WithHandle calls a supplied function f with a valid process handle as an argument.
// The handle is guaranteed to refer to process p until f returns, even if p terminates.
// If no process handle is available, it returns ErrNoHandle.
//
// Currently, process handles are only available on Linux 5.4 and later
// (where it is known as pidfd) and Windows.
func (p *Process) WithHandle(f func(handle uintptr)) error

var ErrNoHandle = errors.New("process handle unavailable")