Go version

go1.25rc3 windows/amd64

Output of go env in your module/workspace:

set AR=ar
set CC=gcc
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_ENABLED=1
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set CXX=g++
set GCCGO=gccgo
set GO111MODULE=
set GOAMD64=v3
set GOARCH=amd64
set GOAUTH=netrc
set GOBIN=E:\Programs\Omion\go
set GOCACHE=C:\Users\Omion\AppData\Local\go-build
set GOCACHEPROG=
set GODEBUG=
set GOENV=C:\Users\Omion\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFIPS140=off
set GOFLAGS=
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\Omion\AppData\Local\Temp\go-build3306884133=/tmp/go-build -gno-record-gcc-switches
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMOD=D:\Documents\git\lsmbackup\go.mod
set GOMODCACHE=C:\Users\Omion\go\pkg\mod
set GONOPROXY=myhost.example.org
set GONOSUMDB=myhost.example.org
set GOOS=windows
set GOPATH=C:\Users\Omion\go
set GOPRIVATE=myhost.example.org
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\Omion\sdk\go1.25rc3
set GOSUMDB=sum.golang.org
set GOTELEMETRY=local
set GOTELEMETRYDIR=C:\Users\Omion\AppData\Roaming\go\telemetry
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\Users\Omion\sdk\go1.25rc3\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.25rc3
set GOWORK=
set PKG_CONFIG=pkg-config

What did you do?

Attempted to use the new 1.25 Windows asynchronous IO on a volume handle via: * windows.CreateFile(..., windows.FILE_FLAG_OVERLAPPED) * os.NewFile(...) * file.ReadAt(...)

The problem seems to be that, for whatever reason, Windows disallows seeking on asynchronous volume handles. This is normally not a problem, since changing the file position is useless on an asynchronous handle (they need an OVERLAPPED structure for I/O, and if an OVERLAPPED structure is specified the file position is not used). However, Go calls syscall.Seek twice in the internal/poll/*FD.Pread function, used for ReadAt.

The fix for this would be to not call syscall.Seek in Pread (and probably also Pwrite - it seems like it would have the same problem) for files opened with FILE_FLAG_OVERLAPPED.

I think removing syscall.Seek could be done (though I'm not sure I understand all edge cases):

  • The two syscall.Seek() in Pread (and similarly in Pwrite) could be removed, and the deferred fd.setOffset(curoffset) could be based off of the initial fd.offset instead of the Seek.
  • poll/*FD.Seek would have to be changed for whence==io.SeekCur, but it could be converted to using whence==io.SeekSet and adding fd.offset to the passed offset. Neither of the other whences would have to change, as they do not use the file position as an input.

I suspect it's not a problem that most people will have. However, my use case requires volume handles, and I was hoping to get rid of my home-built IOCP processor. It would make my code quite a bit cleaner.

I have attached an example file that demonstrates this problem (renamed from .go to .txt to make Github happy). HOWEVER, it must be run as administrator and involves opening raw volumes, which is generally a dangerous thing to do. Caution is advised! nativeasync.txt

What did you see happen?

ReadAt fails with "The parameter is incorrect"

What did you expect to see?

No error returned, like with synchronous handles and asynchronous handles to files.

Comment From: gabyhelp

Related Issues

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: dmitshur

Thanks for the report.

CC @golang/windows.

Comment From: qmuntal

Thanks for reporting and testing this new feature. On vacations right now, don't think I'll have time to work on this before Go 1.25 is released, but will fix it once back.

Comment From: qmuntal

poll/*FD.Seek would have to be changed for whence==io.SeekCur, but it could be converted to using whence==io.SeekSet and adding fd.offset to the passed offset. Neither of the other whences would have to change, as they do not use the file position as an input.

I've created a dedicated issue for this change: #75081.

Comment From: gopherbot

Change https://go.dev/cl/697275 mentions this issue: os: set the correct file offset in File.Seek for Windows overlapped handles

Comment From: qmuntal

I've reproduced the issue locally. I wonder why you want to read from a volume? Which information does it return?

Comment From: gopherbot

Change https://go.dev/cl/697295 mentions this issue: internal/poll: don't call Seek for overlapped Windows handles

Comment From: qmuntal

@dmitshur this issue is a corner-case. I would backport it to Go 1.25 anyway given that it is an easy fix and that there might be other type of overlapped file handles that don't support seeking.

Comment From: omion

I've reproduced the issue locally. I wonder why you want to read from a volume? Which information does it return?

It opens the entire partition like a file (similar to opening something like /dev/sda1 in Linux), letting you do things like make clones of a filesystem.

I use this functionality for a backup program that I have been writing off-and-on for the past few years. Using Windows' volume shadow copy service, I make a snapshot of the volume. Then I open the snapshot and read the entire thing for updating a network-based backup.

One of the computers that I run it on has large and fast SSDs, and I found that using overlapped file handles and I/O completion ports produced the fastest backups. My current system uses a pool of goroutines with runtime.LockOSThread() listening to completion events, and another pool queuing I/O for each of the volumes (I have multiple that all backup in parallel). I have found that using an I/O completion port on handles opened with FILE_FLAG_NO_BUFFERING gives the highest throughput with the lowest CPU usage (important since other parts of the program are quite CPU intensive).

If I could take advantage of the Go runtime's I/O completion port then I could replace my two pools of goroutines-that-are-actually-threads and it would make my code quite a bit cleaner. I could treat individual I/O as synchronous, relying on the runtime to swap goroutines when needed. I wouldn't have to have a hot, shared data structure for the completion port threads to look up when the OS wakes them up randomly, since each I/O would be completely contained within one goroutine. (I'm not 100% sure this is how it would work, since I quickly ran into this issue when I tried to use it, but it should be similar.)

Comment From: qmuntal

@gopherbot please backport to Go 1.25. Support for Windows overlapped handles were introduces in Go 1.25.

Comment From: gopherbot

Backport issue(s) opened: #75111 (for 1.25).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

Comment From: gopherbot

Change https://go.dev/cl/698035 mentions this issue: internal/poll: don't call SetFilePointerEx in Seek for overlapped handles

Comment From: gopherbot

Change https://go.dev/cl/704315 mentions this issue: [release-branch.go1.25] internal/poll: don't call Seek for overlapped Windows handles