In order to use the runtime poller with os.File on Windows, the file or pipe will have to be opened with FILE_FLAG_OVERLAPPED. We shouldn't do that unconditionally unless we can remove it if the program calls the Fd method. Programs currently expect the Fd method to return a handle that uses ordinary synchronous I/O.
Comment From: gopherbot
CL https://golang.org/cl/36800 mentions this issue.
Comment From: rasky
ReOpenFile() can do just that, adding/removing FILE_FLAG_OVERLAPPED (https://msdn.microsoft.com/en-us/library/aa365497(VS.85).aspx)
Comment From: polarina
Overlapped reads on Windows may run synchronously under some conditions. Files using NTFS compression can not be read asynchronously, for example.
See https://support.microsoft.com/en-us/help/156932/asynchronous-disk-i-o-appears-as-synchronous-on-windows
Comment From: ianlancetaylor
Thanks. For our purposes it doesn't matter if I/O is sometimes synchronous. The effect will be that the I/O operation will tie up a thread. That is not ideal, but it is no different than any other blocking syscall.
Comment From: jstarks
There's no way to flip the bit on an existing file handle, and ReOpenFile will only work properly for real local files, only if the file does not have the delete on close bit set (I think), and only if they have been opened with compatible sharing flags. And of course Fd() uintptr does not provide a mechanism to return an error if something goes wrong.
I'm afraid this may be impossible to achieve with the current Windows API.
This is probably not such a shame for normal files opened for cached IO, although recent versions of Windows do perform cached reads asynchronously in some cases. But it certainly is limiting for named pipes, files opened with FILE_FLAG_NO_BUFFERING, and other Windows file handle types that reliably perform IO asynchronously.
Maybe this is something that we can improve in Windows. But even if we do, I think for Windows there would still be a lot of value in providing direct access to the runtime poller. There are several APIs that take OVERLAPPED structures that don't fit into os.File (ConnectNamedPipe, MergeVirtualDisk, DeviceIoControl, etc.), and it would be ideal to be able to use the existing poller to avoid blocking threads on these APIs.
That seems at odds with some of the discussion in #18507, unfortunately.
Comment From: qiulaidongfeng
CL has been merged, is it still necessary to open this issue?
Comment From: qiulaidongfeng
cc @golang/windows This have OS-Windows label , CL has been merged. Is there anything else to be done here?
Comment From: crvv
The merged CL(36800) updated this issue. It didn't fix this issue.
According to the discussion above. This issue can't be fixed because of the Fd method.
My personal opinion is that Fd usages should be replaced by SyscallConn. And Fd can be deprecated and removed.
Comment From: qmuntal
We might not be able to change os.OpenFile to return a handle opened with FILE_FLAG_OVERLAPPED, but we could improve our os.File overlapped (aka async) I/O support for user-provided handles (via os.NewFile). That would certainly be useful for many.
Comment From: gopherbot
Change https://go.dev/cl/660495 mentions this issue: internal/poll: always use SetFileCompletionNotificationModes on non-socket handles
Comment From: gopherbot
Change https://go.dev/cl/660496 mentions this issue: internal/poll,net: set SIO_UDP_CONNRESET in net
Comment From: gopherbot
Change https://go.dev/cl/660595 mentions this issue: internal/poll: support async file operations on Windows
Comment From: gopherbot
Change https://go.dev/cl/661795 mentions this issue: os,internal/poll: support I/O on overlapped handles not added to the poller
Comment From: gopherbot
Change https://go.dev/cl/661955 mentions this issue: internal/poll: defer IOCP association until first IO operation
Comment From: gopherbot
Change https://go.dev/cl/662215 mentions this issue: internal/poll: simplify execIO
Comment From: gopherbot
Change https://go.dev/cl/662236 mentions this issue: os: support overlapped IO with NewFile
Comment From: qmuntal
Cl 662236 will close this issue. It will make os.NewFile support overlapped handles by adding them to the runtime poller, which will allow using deadline on those files.
Notably, os.Open, os.Create, os.OpenFile, and os.Pipe, will remain synchronous, as changing them would be a breaking change. I plan to submit a follow-up proposal to make syscall.Open (used by os.OpenFile) map the O_NONBLOCK flag to FILE_FLAG_OVERLAPPED, which would allow users to create overlapped files without leaving the os package.
Comment From: gopherbot
Change https://go.dev/cl/664455 mentions this issue: os,internal/poll: disassociate handle from IOCP in File.Fd
Comment From: qmuntal
Reopening this issue. I think we can still do a bit better by supporting deadlines for non-overlapped handles.
Comment From: gopherbot
Change https://go.dev/cl/665315 mentions this issue: internal/poll: remove outdated tests
Comment From: gopherbot
Change https://go.dev/cl/668095 mentions this issue: os: test overlapped pipes deadlines on Windows
Comment From: gopherbot
Change https://go.dev/cl/668195 mentions this issue: net,os: support converting between *os.File and net.Conn on Windows
Comment From: database64128
Now that we can convert between *os.File and net.Conn types, the only missing piece seems to be syscall.RawConn support, which probably requires proposals for new types and methods. Any plans for that?
Comment From: qmuntal
Now that we can convert between *os.File and net.Conn types, the only missing piece seems to be syscall.RawConn support, which probably requires proposals for new types and methods. Any plans for that?
Yes, it's on my TODO list for Go 1.26 (it's already too late for Go 1.25).