Background

59149 removed the package restrictions on the use of `go:wasmimport`, but established strict constraints on the types that can be used as input and result parameters. The motivation for this was that supporting rich types between the host and the client would require sophisticated and expensive runtime type conversions because of the mismatch between the 64 bit architecture of the client and the 32 bit architecture of the host.

With the upcoming 32 bit wasm port, this problem goes away, as both client and host will use 32 bit pointers. However, we can also support a limited set of types on 64 bit platforms, where no runtime conversion is necessary.

Proposal

Relax the constraints on types that can be used as input and result parameters with the go:wasmimport compiler directive. The exact allowed types would depend on whether wasm or wasm32 is used.

We define the "small integer types" group as the set of types described by [u]int[8|16]. The following types would be allowed as input parameters to any wasmimport/wasmexport function:

bool
int32, uint32, int64, uint64
float32, float64
string
uintptr, unsafe.Pointer, *T where T is an allowed parameter type or one of the small integer types.
*struct. All fields of the *struct must be allowed parameter types, struct, [...]T, or one of the small integer types.
- If the struct has any fields, it must embed structs.HostLayout (see #66408).
- Any struct fields must also embed structs.HostLayout (recursively).
- *T, unsafe.Pointer, uintptr, and string types are only allowed in *struct on wasm32.
*[...]T where T is an allowed type or one of the small integer types.

All input parameter types except string are also allowed as result parameter types.

The following types would remain disallowed as input and output parameter types:

chan T
complex64, complex128
func
interface
map[T]U
[]T
struct
[...]T

The conventions established for use of pointers in CGO will be required when using pointers with wasmimport/wasmexport, e.g. the host can read Go memory, can write pointerless data (like the content of a byte buffer) but cannot write Go pointers to Go memory, and cannot hold on to Go pointers unless they are pinned.

Discussion

Compatibility guarantees

The Go spec does not specify the struct layout and leaves it up to implementations to decide. As such, we cannot provide a guaranteed ABI without having to change the spec or force future layout changes to provide runtime conversion of data. This proposal suggests making it clear to users through documentation that there are no guarantees of compatibility across versions of the Go compiler.

Type conversion rules

The following conversion rules would be automatically applied by the compiler for the respective parameter type:

Go Type	Parameter type (per Wasm spec)
`bool`	`i32`
`int32, uint32` `int64, uint64`	`i32, i32` `i64, i64`
`float32, float64`	`f32, f64`
`string`	Assigned to two call parameters as a `(i32, i32)` tuple of (pointer, len). Only allowed for input parameters.
`uintptr`, `unsafe.Pointer`, `T`, `struct`, `*[...]T`	`i32, i32, i32, i32, i32`

Strings

Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter.

Supporting `GOARCH=wasm`

The wasm architecture uses 64 bit pointers and integer sizes. As the host uses 32 bit pointers, this makes it impossible to allow certain types without costly runtime conversions, such as *struct types containing pointer fields. Since string types are also pointer types, *struct types containing string fields are also disallowed.

Supporting `[u]int`, `[u]int8`, `[u]int16` as concrete parameters

The [u]int types are problematic as the size of them are not precisely defined, and may cause confusion when used with strictly 32 bit or 64 bit integers. The [u]int8 and [u]int16 types are problematic because we would be forced to automatically convert them to/from the i32 wasm representation, with potential loss of precision or overflow. They are still allowed as pointer type, array elements and struct fields.

Supporting slices, maps

Both slices and maps are disallowed because of the uncertainty around the memory underlying these types and interactions with struct and array rules. Users who wish to use slices can manually use (&slice, len(slice)) or unsafe.Pointer. There is no clear way to support passing or returning map data from the host other than by using unsafe.Pointer and making assumptions about the underlying data.

Related proposals

struct.Hostlayout

66408 proposes a way for users to request that struct layout is host compatible. Our proposal depends on the definitions put forward in this proposal for `struct` parameters.

Future work

WASI Preview 2 (AKA WASI 0.2)

WASI Preview 2 defines its API in terms of the Component Model, with a rich type system and an IDL language, WIT. The Component Model also defines a Canonical ABI with a specification for lifting and lowering Component Model types into and out of linear memory. This proposal does not attempt to define the ABI for any hypothetical wasip2 target, and would leave such decisions for any future wasip2 proposal.

Supporting `struct` and `[...]T` by value

A previous version of this proposal included support for passing struct and [...]T types by value by expanding each field recursively into call parameters. This was removed in favor of a simpler initial implementation but could be re-added if users require it.

Contributors

@johanbrandhorst, @evanphx, @achille-roussel, @dgryski, @ydnar

CC @cherrymui @golang/wasm

@gabyhelp's overview of this issue: https://github.com/golang/go/issues/66984#issuecomment-2593973519

Comment From: dr2chase

"This follows the C struct value semantics" is just a hair vague; are 8-byte quantities (float64, int64, uint64) stored at a 4-byte or 8-byte alignment? It was my understanding (and the purpose of #66408) to specify a ~~4-byte~~ alignment for fields of those types when they occur in structs passed to wasm32 (tagged structs.HostLayout).

(edited to note error, the host alignment for 8-byte integers and floats is 8 bytes).

Comment From: ydnar

Ideally 8-byte values would always be 8-byte aligned in the wasm32 port.

Comment From: evanphx

@dr2chase Looking at what clang does, it uses 8-byte alignment on 64bit quantities so we'd match that.

Comment From: dr2chase

You are right, I got it backwards. But that is what you are expecting for anything that has pointers-to-it passed to the wasm host platform, yes?

Comment From: cherrymui

Thanks for the proposal! A few questions: - 8-byte alignment for 64-bit values, as mentioned above. #63131 doesn't seem to have a definitive answer, and currently it seems the CL doesn't implement 64-bit alignment. - If we don't always align 64-bit value to 8 bytes (which differs from current all 32-bit architectures we support and probably requires quite some work), we should align 64-bit value to 8 bytes when structs.HostLayout is specified. So #66408 is very related. - structs and arrays. What is the ABI specification exactly? The C ABI on, say ELF AMD64, is pretty complex for passing structs and arrays. Small fields may be packed into one word. Large structs may be passed indirectly (stored on stack, passing a pointer to the callee). Do we have a specification for this? - string. What does a string look like on Wasm/WASI side? I couldn't find its specification on WASI P1 doc. On Component Model doc https://github.com/WebAssembly/component-model/blob/main/design/mvp/CanonicalABI.md (which I guess is for WASI P2, not P1?), it specifies string is two i32, which is similar to Go's string, which is good. But also it allows three encodings, UTF-8, UTF-16, and "latin1+utf16" differentiated by a high bit. The second and third encoding are not compatible with Go strings. Do we require UTF-8 encoding? Or we don't allow passing Go strings directly?

Besides, for structs, arrays of structs, and pointer to structs, I would suggest we allow only structs with structs.HostLayout to be passed. The reason is that in the Go spec we don't require struct fields to be laid out in memory in source order, and it may well change in a future Go release. structs.HostLayout specifies a fixed layout. Structs without that marker can change. This gives a clear way to say which structs should have a fixed layout, which are okay to change.

Thanks.

Comment From: dr2chase

Two other questions, first:

type w32thing struct {
    _ structs.HostLayout
    a uint8
    b uint16
}

Is this laid out a_bb or is it aaaabbbb? What sizes do I use for struct fields? I assume it is the smaller ones, but I wanted to verify this else it would be a problem.

Second, passing pointers to 8-byte primitive types to the host will be tricky unless those references come from fields in structures tagged with HostLayout -- otherwise, they may not be aligned. So

type wx struct {
   _ structs.HostLayout
  x int64
}
func f(x int64, w wx) {
  someWasmFunc(&x) // might not work, x might not be 8-byte aligned
  someWasmFunc(&w.x) // this will work because w is a wx and its x field is 8-byte aligned
  someOtherWasmFunc(&w) // if it used *wx for its parameter type instead of *int64
}


**Comment From: johanbrandhorst**

Thanks for the quick feedback! I've tried to answer each question:

> structs and arrays. What is the ABI specification exactly? The C ABI on, say ELF AMD64, is pretty complex for passing structs and arrays. Small fields may be packed into one word. Large structs may be passed indirectly (stored on stack, passing a pointer to the callee). Do we have a specification for this?

The specification falls out of the table of transformations (I think?). There current plan isn't to introduce any sort of magic around large structs or field packing. Structs fields are added as call parameters, from the first field to the last, according to the conversion rules for the type of the field. Examples:

```go
type foo struct {
    a int
    b string
    c [2]float32
}

With a function signature of

//go:wasmimport some_module some_function
func wasmFunc(in foo) int

Would roughly translate to (in WAT format)

// $a is of type `i32` holding the value of `a`
// $b_addr is of type `i32` and is a pointer to the start of the bytes for the Go string `b`
// $b_len is of type `i32` and is the length in bytes to read from `$b_addr` to get the whole string
// $c_0 is of type `f32` and is the value of `c[0]`
// $c_1 is of type `f32` and is the value of `c[1]`
call $some_function (local.get $a) (local.get $b_addr) (local.get $b_len) (local.get $c_0) (local.get $c_1)

Struct fields would be expanded into call parameters before subsequent fields at the same level.

What does a string look like on Wasm/WASI side?

For wasip1, we will treat Go string parameters simply as a (*byte, int) tuple. There will be no encoding constraints, just as with regular Go strings. To the Wasm host, it will look identical to using struct { a *byte; b int } as a parameter. For wasip2, those constraints would have to be considered in a hypothetical future wasip2 proposal.

Making structs.HostLayout required for structs, arrays of structs and pointers to structs

This sounds like a great idea, and we should also extend it to pointers to 8 byte sized primitive types to guarantee alignment, as suggested by @dr2chase's last question. This would avoid any question around alignment issues for pointers. It hurts the ergonomics a little bit but that's a price worth paying, I think.

type w32thing struct { _ structs.HostLayout a uint8 b uint16 }

Is this laid out a_bb or is it aaaabbbb? What sizes do I use for struct fields? I assume it is the smaller ones, but I wanted to verify this else it would be a problem.

I'm a little confused by the question to be honest. If this type was used as an input to a Wasm call, it would look like this:

// $a is of type `i32`
// $b is of type `i32`
call $some_function (local.get $a) (local.get $b)

I suppose that might mean the memory looks like this: a___bb__? We're not passing a pointer to the struct or the fields, so we'd need to copy the values into locals, which will be of type i32 (I think)? Admittedly my grasp of this exact part of the code is a bit weak so I appreciate corrections.

Comment From: cherrymui

Thanks for the response!

Structs fields are added as call parameters, from the first field to the last, according to the conversion rules for the type of the field.

This sounds like a reasonable choice. Is this ABI specified anywhere in Wasm/WASI docs? Or the Wasm side has to define the function taking parameters element-wise?

For wasip1, we will treat Go string parameters simply as a (byte, int) tuple. There will be no encoding constraints, just as with regular Go strings. To the Wasm host, it will look identical to using struct { a byte; b int } as a parameter.

This sounds reasonable as well. Is it specified anywhere in Wasm/WASI docs?

Thanks.

Comment From: johanbrandhorst

This sounds like a reasonable choice. Is this ABI specified anywhere in Wasm/WASI docs? Or the Wasm side has to define the function taking parameters element-wise?

I don't know about this being an official ABI so much as just a consequence of the Wasm spec around function calls and how we can apply Go semantics to it. We're limited to the i32, i64, f32 and f64 value types, and the call instruction takes a function index and arguments from the stack. In order to simulate pass-by-value for structs, we have to flatten each field to one of the allowed value types.

This sounds reasonable as well. Is it specified anywhere in Wasm/WASI docs?

Not sure there's a doc anywhere, but practically, definitions like path_create_directory, which take a string parameter, use this pattern: https://cs.opensource.google/go/go/+/refs/tags/go1.22.2:src/syscall/fs_wasip1.go;l=230.

Comment From: dr2chase

I guess my question is whether a pointer-to-struct is ever passed from Go to the WASM platform, and therefore, what expectations the WASM side has about the layout of the fields of that structure. structs.HostLayout is intended to obtain those expectations, but (1) do we even need to do this? We thought we did, and (2) we need to know what the expectations are. I think it was just that 64-bit floats and ints get 64-bit alignment.

I don't think this is for specifying the layout that gets passed to a WASM call if the struct is passed by value.

Comment From: cherrymui

I don't know about this being an official ABI so much as just a consequence of the Wasm spec around function calls and how we can apply Go semantics to it. We're limited to the i32, i64, f32 and f64 value types, and the call instruction takes a function index and arguments from the stack. In order to simulate pass-by-value for structs, we have to flatten each field to one of the allowed value types.

As the ABI doesn't have a way to pass struct by value, do we need to support it? If users on the other (non-Go) side have to define the function as taking arguments element-wise with primitive types and pointers, it would probably be better to define the same way on the Go side. Does any other language have a Wasm/WASI interface that allows passing struct by value?

(Same applies for arrays. Pointer to struct/array is fine.)

Comment From: johanbrandhorst

I guess my question is whether a pointer-to-struct is ever passed from Go to the WASM platform, and therefore, what expectations the WASM side has about the layout of the fields of that structure.

I think the biggest concern around this is that all 64 bit values use 8 byte alignment, as you say. We definitely want this, so I think that on its own makes the case for structs.HostLayout. For other values, I think we want to just use "natural alignment" (4 byte for 4 byte values, etc). As far as we know, there is no strict enforcement of this in Wasm generally, but this is the approach taken by LLVM, so it probably makes sense for us to keep it the same.

I also don't know that it's an important question for this proposal in particular, since the answer is pretty clear regarding what we should be passing in the call instruction when encountering a pointer (an i32). I'm happy to weigh in on #66408 if needed to have this discussion though.

As the ABI doesn't have a way to pass struct by value, do we need to support it?

It's fair to say that we can just not support structs and arrays by value, their use are likely to be limited (why not use a pointer?), and it would significantly simplify the implementation. We can come back to it if we need to later. I'll update the proposal.

Comment From: aykevl

On the TinyGo side we're working on an implementation of this proposal, so here's my perspective on it from TinyGo:

Always passing structs by reference (pointer) seems like a good idea. Passing structs by value gets complicated quickly and if we agree on something in the future it's easy enough to add it in a new proposal. Structs in memory are relatively well defined in comparison.
Strings in TinyGo are already of the format specified in this proposal, so that's nice. We won't need to do anything special there.
TinyGo uses the host layout (it's based on LLVM) so TinyGo needs no struct.HostLayout pragma. But it would be relatively simple to add a check like that to ensure compatibility, and if that is what the proposal ends up with I'll make sure we will have the same strict checking.
TinyGo has always had a 32-bit wasm implementation (int, uintptr and pointers are 32-bit). Therefore, it would make sense to allow these values at all times. That's a possible compatibility concern, but in essence we're already incompatible so I'm not sure how much of an issue this is. Thoughts?

Question: what fields would be allowed in these structs? I would assume a struct with a chan field would be disallowed, for example. This isn't part of the proposal yet though, so perhaps this can be added? Something like this:

Structs may not be passed by value, but pointers to structs are allowed. Every field in a struct must be one of the allowed parameter types, or be a struct (recursively).

Comment From: cherrymui

TinyGo has always had a 32-bit wasm implementation (int, uintptr and pointers are 32-bit). Therefore, it would make sense to allow these values at all times.

I think this is fine. And we should allow them in Go gc toolchain for the "wasm32" port.

Structs may not be passed by value, but pointers to structs are allowed. Every field in a struct must be one of the allowed parameter types, or be a struct (recursively).

Yeah, something along this line makes sense. And also for arrays. I'd say a struct field or a struct pointed by a field should also have the HostLayout marker (because the marker is not recursive).

Comment From: johanbrandhorst

Thanks for your thoughts Ayke, it's always appreciated.

TinyGo has always had a 32-bit wasm implementation (int, uintptr and pointers are 32-bit). Therefore, it would make sense to allow these values at all times. That's a possible compatibility concern, but in essence we're already incompatible so I'm not sure how much of an issue this is. Thoughts?

As Cherry says, these values will be allowed since this proposal is restricted to the wasm32 architecture. The wasm architecture will not have these new relaxations applied. I'm not sure I understand the incompatibility?

Structs may not be passed by value, but pointers to structs are allowed. Every field in a struct must be one of the allowed parameter types, or be a struct (recursively).

I'd say a struct field or a struct pointed by a field should also have the HostLayout marker (because the marker is not recursive).

I've added some clarifying words to the proposal, please take a look!

Comment From: aykevl

The updated proposal looks good to me! (If I'm very pedantic, it doesn't explicitly say that a struct in a struct is allowed, though it clearly should be. Right now it says *struct is allowed but struct isn't).

However, I have to say that @ydnar has pointed out that the Canonical ABI also allows structs, and it would be nice to have them supported in //go:wasmimport. That said, if I'm reading the specs correctly, the Canonical ABI and the C ABI are incompatible when it comes to structs: the C ABI passes structs by value only when it contains only one field after flattening, while the Canonical ABI passes records (similar to structs) by value if the number of fields is 16 or less after flattening. So that means //go:wasmimport would have to choose between the C ABI and the Canonical ABI.

As Cherry says, these values will be allowed since this proposal is restricted to the wasm32 architecture. The wasm architecture will not have these new relaxations applied. I'm not sure I understand the incompatibility?

Nevermind, TinyGo doesn't even support GOOS=js GOARCH=wasm tinygo ..., it just uses tinygo -target=wasm. So in essence tinygo -target=wasm ... is equivalent to GOOS=js GOARCH=wasm32 go .... Basically it has always been a GOOS=wasm32 implementation and never supported what would be GOOS=wasm (with 64-bit pointers).

I'd say a struct field or a struct pointed by a field should also have the HostLayout marker (because the marker is not recursive).

Seems like a good idea. It's easier to remove such a restriction in the future (if it turns out to be unnecessary) than it is to introduce it later. But I don't know Go internals well enough to say it is needed.

Comment From: johanbrandhorst

To comment quickly on the Canonical ABI: it doesn't relate to this proposal directly as this proposal only targets the wasip1 port, and the Canonical ABI is a preview 2 document (as far as I know). A hypothetical wasip2 proposal would have to tackle type constraints for go:wasmimport (and go:wasmexport) as they relate to the Canonical ABI.

Comment From: cherrymui

I'm okay with supprting passing structs by value if there is a widely used ABI that is not too complex (if it is as complex as the ELF C ABI on amd64, I'm not sure). If currently there is no widely agreed ABI for structs, we can wait. We can always add things later.

Comment From: ydnar

Hi, original author of the relaxed type constraints proposed here.

The TinyGo PR where this originated depends on LLVM to flatten structs and arrays. This works in practice most of the time, except when it doesn't: namely the Component Model and WASI 0.2 extensively uses tagged unions (variant types in WIT).

The code generator (wit-bindgen-go) implements the flattening rules as specified in the Canonical ABI, which then leans on LLVM to flatten the Go structs that represent variant types.

The CABI flattening rules are per-field, so if a variant has a case that includes a 64-bit wide field, then the flattened representation of the variant must use an i64 at that position.

Given that the compiler is ignorant of the CABI layout, this strategy cannot correctly represent these variant types when passed by value.

@cherrymui: LLVM does correctly flatten structs and arrays consistent with the CABI spec (my sense is the former informed the latter). If we want to start with a more constrained set of types now and relax later, we can make that work.

Comment From: aykevl

LLVM does correctly flatten structs and arrays consistent with the CABI spec (my sense is the former informed the latter).

Not exactly. If you pass a LLVM struct like {i32, i32} by value, LLVM will happily flatten the struct and pass it as values. But if you do that in C, Clang will pass the struct by reference, not by value: it will reserve some space on the stack and pass a pointer instead. See: https://godbolt.org/z/YjKj5o3c4

I believe this is why the Component Model lowers everything to bare i32/i64/f32/f64/pointer values in function signatures, which have no ambiguity in what ABI they should have on the WebAssembly level.

Comment From: ydnar

With the exception of the check for structs.HostLayout, I’ve implemented the rules as defined in this proposal here in TinyGo: https://github.com/dgryski/tinygo/pull/16/commits/6921ce6520f8dc6247ccccfa9d094820df33174b

Initial (shallow) tests against WASI 0.2 APIs seem to work.

Comment From: johanbrandhorst

I've updated the proposal to explicitly state that [...]T are allowed as struct fields when a pointer of that type is used as an input parameter.

Comment From: ydnar

Given that unsafe.Pointer is allowed as a result type, and *T is allowed as a parameter type, I think it's reasonable to expect pointers to Go-managed memory to pass between guest and host, particularly with the addition of go:wasmexport.

Given this, I propose relaxing the result types to allow *T as well.

This would result in more type-safe interfaces. It would also be more or less symmetric with parameter types, with the exception that string would not be permitted as a return type, as it decomposes into a pair of uintptr.

Comment From: johanbrandhorst

Given that unsafe.Pointer is allowed as a result type, and *T is allowed as a parameter type, I think it's reasonable to expect pointers to Go-managed memory to pass between guest and host, particularly with the addition of go:wasmexport.

Given this, I propose relaxing the result types to allow *T as well.

To paraphrase some discussions we've had on this, there are some outstanding questions before I think we can make this change, namely:

Allowing *T result parameters for go:wasmimport functions would make it (probably?) impossible for the GC to track the underlying memory without making assumptions about how it was created. E.g. even if we could assume that any *T returned are allocated by a function exported by Go for the purposes of allocating memory within its memory space, how do we map that allocation (probably a []byte) to the *T, such that the GC can track the lifetime of the *T and free the memory as appropriate? We either end up freeing the []byte before we're done with the *T or end up tracking the []byte as in use forever while freeing the *T when it's done.
Even allowing *T result parameters for go:wasmexport functions might not be safe, as the GC may assume that a piece of memory is no longer in use once we have returned from the function execution, though it probably shouldn't run until another export is called, so it may be safe until another export is called.

My primary motivation behind this change is to allow better ergonomics when using Wasm without compromising on safety. If the argument for allowing *T is that unsafe.Pointer is allowed, I think there is clearly a difference in user expectations when using *T and using unsafe.Pointer, and it's important that we only allow *T if we can do so safely. Unless we are confident we can allow using *T as a result parameter safely, including if we have to restrict it to be used with wasimport or wasmexport only, I think we should wait to include it.

Comment From: johanbrandhorst

Are there any outstanding concerns for this proposal?

Comment From: rsc

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

Comment From: cherrymui

I think the updated proposal is pretty reasonable.

Type passed to host | Type read from host

What exactly do they mean? One for wasmimport, one for wasmexport? One for parameter, one for result? It would be good to specify more clearly. I also don't understand why they are not the same. E.g. string is allowed as a "Type passed to host", but not a "Type read from host". Would it mean that the host can only read a string, but cannot construct a string?

For passing pointers between the Go Wasm module and host w.r.t the GC, it is similar to passing pointers in cgo. Cgo has pointer passing rules https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers . Basically

Go code may pass a Go pointer to C provided the memory to which it points does not contain any Go pointers to memory that is unpinned. ...

C code may keep a copy of a Go pointer only as long as the memory it points to is pinned.

Maybe we want to have similar rules?

Also, if we allow unsafe.Pointer which is allowed to point to Go memory, it is not that different from allowing a *T. So maybe we want to have the same allowance (and restriction) for unsafe.Pointer?

GOARCH=wasm

The major difference between wasm and wasm32 is the size of int and pointers. It is probably reasonable to not allow types containing them. Smaller scalers (bool, uint8, etc.) are the same on both, so we probably could allow them on GOARCH=wasm (translate to Wasm i32 the same way).

Comment From: ydnar

The major difference between wasm and wasm32 is the size of int and pointers. It is probably reasonable to not allow types containing them. Smaller scalers (bool, uint8, etc.) are the same on both, so we probably could allow them on GOARCH=wasm (translate to Wasm i32 the same way).

If one authors a wasm function with an int (or a better example uintptr), I expect they know what they're doing (e.g. somewhat similar argument for supporting *T).

Perhaps omit int/uint and permit uintptr?

Comment From: johanbrandhorst

I think the updated proposal is pretty reasonable.

Type passed to host | Type read from host

What exactly do they mean? One for wasmimport, one for wasmexport? One for parameter, one for result? It would be good to specify more clearly. I also don't understand why they are not the same. E.g. string is allowed as a "Type passed to host", but not a "Type read from host". Would it mean that the host can only read a string, but cannot construct a string?

Great question. "Type passed to host" means passed from the Go memory space to the host memory space, and vice versa for "Type read from host". The asymmetry arises from the desire to have the Go GC know about and own all memory pointed to. Passing a *T into a wasmexport function, or returning a *T from a wasmimport function would mean the Go GC has a pointer value without knowing where the original memory was allocated. We did consider having some sort of automated mapping function for a potential exported alloc that the host could call to allocate memory, but it's not clear to me how we could map that to values returned from the host. I'm also not familiar with the exact rules for this in CGO, but when writing this proposal we assumed that uncontrolled memory would be undesirable. I'm happy to relax that constraint if we can make it work.

Strings are unsupported as wasmimport result parameters because we only allow a single result parameter, and strings are synthesized using a pointer and a len tuple. Conversely, they are disallowed as wasmexport input parameters because it would mean passing host allocated memory into Go.

I've renamed the table headings to "Export result/Import parameter" and "Export parameter/Import result". Does that make it clearer?

For passing pointers between the Go Wasm module and host w.r.t the GC, it is similar to passing pointers in cgo. Cgo has pointer passing rules https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers . Basically

Go code may pass a Go pointer to C provided the memory to which it points does not contain any Go pointers to memory that is unpinned. ... C code may keep a copy of a Go pointer only as long as the memory it points to is pinned.

Maybe we want to have similar rules?

Yes, the rules regarding pinning look great. I read through that whole article and still couldn't quite make sense of what it means for C allocated pointers passed to Go functions (analogous to host allocated pointers passed to wasmexport functions) or C allocated pointers returned from C functions (analogous to host allocated pointers return from wasmimport functions). If we can make it work for C we should be able to make it work the same for wasmimport and wasmexport though, I think.

Also, if we allow unsafe.Pointer which is allowed to point to Go memory, it is not that different from allowing a *T. So maybe we want to have the same allowance (and restriction) for unsafe.Pointer?

To me, there's a great difference between these two. One of them has the user deliberately opting in to manual memory management, while the other looks like real, safe Go code. If we can't provide the usual memory guarantees (and I don't see how we can do that with host allocated memory), we shouldn't allow *T, in my opinion.

GOARCH=wasm

The major difference between wasm and wasm32 is the size of int and pointers. It is probably reasonable to not allow types containing them. Smaller scalers (bool, uint8, etc.) are the same on both, so we probably could allow them on GOARCH=wasm (translate to Wasm i32 the same way).

Perhaps omit int/uint and permit uintptr?

This proposal is limited to the wasm32 architecture. There's no ambiguity in integer and pointer sizes, so we shouldn't need to disallow int or uint. uintptr is allowed both as input and result parameter.

We could expand this proposal to include a type relaxation for the wasm architecture, but I think that'd be better served in a separate proposal, and TBH I don't think we want to encourage people to use wasip1/wasm once wasip1/wasm32 is available, so I'm OK with the asymmetry in ergonomics.

Comment From: cherrymui

Thanks for the reply!

I'm not really sure we need to distinguish "to host" and "from host". With both wasmimport and wasmexport, one can e.g. pass a Go pointer from an argument of wasmimport back to Go as an argument of wasmexport. I think that should be fine. Of course, some types are just impossible, like string as a result (for both wasmimport and wasmexport).

For the same reason, passing a *T as an argument to wasmexport seems okay if it is from an argument of wasmimport.

Of course, there is still an issue about memory safety -- what the host can do for Go managed memory. I think this is very similar to cgo, and we probably want to apply similar restrictions, e.g. the host can read Go memory, can write pointerless data (like the content of a byte buffer) but cannot write Go pointers to Go memory, and cannot hold on to Go pointers unless they are pinned. Even with the distinction between "to host" and "from host", I think we still need the restrictions, e.g. it is allowed to pass a Go pointer as an argument of wasmimport to the host, but the host should not hold on to it after the call has returned.

Comment From: johanbrandhorst

That sounds good to me, the rules around CGO have worked fine for CGO so we can relax some static guarantees with the use of convention. I've updated the proposal to remove the distinction between passing to the host and reading from the host, and also added a section to the proposal about relying on the CGO conventions for safe use.

Comment From: cherrymui

The updated proposal looks good. Thanks!

For consistency, it may still be reasonable to apply similar relaxations on GOARCH=wasm. wasip1/wasm is probably not going to go away very soon (especially given that wasm32 isn't available yet). - For most primitive types (bool, int8, uint8, int16, uint16, int32, uint32, int64, uint64) it is straightforward. - int and uint are 64-bit on GOARCH=wasm, so they probably should be converted to Wasm i64. Or we continue disallowing them for now. - Pointers are 64-bit on GOARCH=wasm, but we convert to 32-bit on the host side. We currently allow unsafe.Pointer and apply the conversion. We could also allow T and do the same conversion. - For struct, if it contains a pointer field, the field would be 64-bit on the Go side and 32-bit on the host side, so there is a mismatch. HostLayout probably should not shrink the field size, either. So I think we'll disallow structs with pointers on GOARCH=wasm. - For *struct containing only allowed fields and no pointers, we can allow it if it embeds HostLayout. - strings could be allowed if directly used as arguments or results. We apply the same conversion to (i32, i32) on the host side. Strings contains pointers, so for the reason above string-containing structs are not allowed.

Comment From: johanbrandhorst

Thanks for the thorough summary, I agree that this would clearly provide value to users. I've updated the proposal to detail the aim to support both wasm and wasm32. For now, we allow precise primitive types (excluding int and uint), and *struct not containing pointer or string fields. Just as on wasm32, struct types and any struct fields must recursively embed HostLayout.

Comment From: rsc

Based on the discussion above, this proposal seems like a likely accept. — rsc for the proposal review group

The proposal is https://github.com/golang/go/issues/66984#issue-2257908454

Comment From: aclements

I think this is looking really good. A few thoughts:

I'm slightly concerned about allowing int and uint. The argument is that on wasm32, they're 32 bits in Go, so can be safely passed as i32, but the size of int and uint aren't specified by the language (the spec only says that they can be 32 or 64 bits). Of course, the platform is what defines the size of these, so I could see an argument that wasm32 defines them to be 32 bits and thus interchangeable as an i32. But by the logic of that argument, wasm should define them to be 64 bits and thus interchangeable as an i64.
What happens if the user passes a *struct containing a smaller integer type like uint8? If uint8 is passed as a parameter, the proposal defines that it gets expanded to an i32, but we can't change the size of the field in the struct. Is wasm prepared to receive a pointer to a struct containing a field smaller than 4 bytes?
If Go calls a wasm function that returns, say, a Go uint8, this proposal forces a narrowing from wasm's i32 result value to the Go uint8. (There's a symmetric case with wasm passing an i32 to a Go function that takes a uint8.) In regular Go code, we always require an explicit conversion to convert between different size integer types. It seems like we should follow that same design principal here, which would suggest that [u]int8 and [u]int16 should not be allowed as parameter types.

Comment From: ydnar

I think this is looking really good. A few thoughts:

I'm slightly concerned about allowing int and uint. The argument is that on wasm32, they're 32 bits in Go, so can be safely passed as i32, but the size of int and uint aren't specified by the language (the spec only says that they can be 32 or 64 bits). Of course, the platform is what defines the size of these, so I could see an argument that wasm32 defines them to be 32 bits and thus interchangeable as an i32. But by the logic of that argument, wasm should define them to be 64 bits and thus interchangeable as an i64.

If developer chooses to use int, uint or uintptr in a wasmimport or wasmexport signature, I think we can assume they know what they’re doing, and expect a 32-bit or 64-bit representation depending on GOARCH.

Edit: this proposal constraints int and uint to GOARCH=wasm32, so until there’s wasm64, I think this is moot?

What happens if the user passes a *struct containing a smaller integer type like uint8? If uint8 is passed as a parameter, the proposal defines that it gets expanded to an i32, but we can't change the size of the field in the struct. Is wasm prepared to receive a pointer to a struct containing a field smaller than 4 bytes?

The struct isn’t directly passed over the ABI, so it doesn’t matter. That’s where structs.HostLayout comes in. Some context: https://github.com/golang/go/issues/63131#issuecomment-1925945414

Comment From: aclements

Edit: this proposal constraints int and uint to GOARCH=wasm32, so until there’s wasm64, I think this is moot?

Since I'm not deep in wasm, it took me a bit to understand your argument here. I think the key context is that we've defined GOARCH=wasm to be "32-bit wasm external interfaces, but it looks like a 64-bit CPU to Go programs" and int and uint are a place where these conflict, so we define that they cannot be passed on GOARCH=wasm. Is that right?

The struct isn’t directly passed over the ABI, so it doesn’t matter. That’s where structs.HostLayout comes in. Some context: https://github.com/golang/go/issues/63131#issuecomment-1925945414

Thanks. The struct layout is still part of the ABI, but you're right that HostLayout will tell Go to use whatever the wasip2 ABI for struct layout is, including for these smaller types (which happens to align with Go's current layout rules except on 64-bit types).

Comment From: johanbrandhorst

A bit late to the latest discussion, but I'm happy to restrict the use of int and uint on both GOARCH=wasm and GOARCH=wasm32 to avoid any ambiguity. We've already disallowed it on GOARCH=wasm. I've updated the proposal.

Comment From: johanbrandhorst

I missed the question about [u]int8 and [u]int16, I think that's also fair (disallowing them to avoid confusion). We can come back to these if the prove to be useful to the community.

Comment From: ydnar

Removing [u]uint8 and [u]int16 may have had the unintended side-effect of disallowing a common use case: passing *byte to a wasmimport call.

The constraints for *T limit T to allowed types, or structs that contain only allowed types. By removing [u]int8 and [u]int16 from the allowed type list, *T where T is [u]int8 or [u]int16 is no longer allowed. Nor would structs containing these types. Is that our intention here?

Also, how should the proposal deal with string (a tuple of *uint8 and uintptr)?

Comment From: johanbrandhorst

Great points Randy, I've updated the proposal to allow [u]int[8|16] as pointer types and struct fields. That will avoid the implicit conversion necessary to support these types as concrete parameters while allowing their use for structs and pointers.

Comment From: aclements

I think *[...]T where T is an allowed type. should also allow [u]int{8,16}. It might be worth just giving a name to the set of types that are allowed via indirection.

Comment From: johanbrandhorst

Thanks, I defined the small integer types group and made it allowed for array elements.

Comment From: ydnar

Can we consider limiting the structs.Hostlayout requirement to structs with > 1 field? This would allow:

structs with fields of type struct{}
structs that embed a single struct with structs.HostLayout

Comment From: johanbrandhorst

I don't see a problem with allowing *struct{} without embedding structs.HostLayout. Not sure about structs with a single field. I've updated the proposal to allow *struct{} without the embedding.

Comment From: aclements

Thanks for the updates!

One nit, in "Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter." the link points to HEAD, which has drifted. It looks like you meant to link to https://go.googlesource.com/go/+/refs/tags/go1.23.0/src/cmd/internal/obj/wasm/wasmobj.go#219.

Comment From: johanbrandhorst

Thanks for the updates!

One nit, in "Strings are not allowed as result parameters as Wasm practically does not allow more than 1 result parameter." the link points to HEAD, which has drifted. It looks like you meant to link to https://go.googlesource.com/go/+/refs/tags/go1.23.0/src/cmd/internal/obj/wasm/wasmobj.go#219.

Thanks for the thorough check, updated 😄

Comment From: aclements

The proposal is https://github.com/golang/go/issues/66984#issue-2257908454. Have all remaining concerns about this proposal been addressed?

Comment From: cherrymui

Thanks for the discussion and update, @johanbrandhorst , @ydnar , and @aclements !

bool is also somewhat similar to the "small integer types", in that it is i32 on the Wasm side (although converting a bool to a (u)int32 needs more code than a simple conversion expression). Should we treat bool one of the "small integer types"? Or is it too inconvenient to disallow bool to be passed directly?

Comment From: ydnar

I can see reasonable arguments for and against allowing small integers and bool values to be included in the list of allowed param types.

The conversion between i32 and bool values is straightforward and can be documented, as can the expansion or truncation of small integers and i32.

LLVM handles this for TinyGo today, so we have a pattern to follow. We have working code that uses bool today (as uint8 either 0 or 1).

If simplifying this proposal is preferred, maybe removing bool from allowed param list is acceptable.

Comment From: aclements

My sense is that bool is okay. My main concern with int8/int16 was that we'd be performing implicit widening and narrowing, which seemed against Go's general requirement that any change in the domain of a numeric value requires an explicit conversion. With bool <-> i32, they're entirely different kinds, rather than just different domains of the same kind, so it doesn't seem likely a programmer would accidentally pass a number instead of a boolean.

Comment From: ydnar

I implemented the current version of this spec in TinyGo, and found one issue in an upstream repo, fixed here (uint → uintptr).

Edit: Another way to address this could be to explicitly declare that the size of int and uint always equal the size of uintptr on wasm32, which would allow int and uint as valid types in this proposal. This would be consistent with how string is explicitly permitted (which implicitly permits int).

Comment From: aclements

No change in consensus, so accepted. 🎉 This issue now tracks the work of implementing the proposal.

The proposal is https://github.com/golang/go/issues/66984#issue-2257908454

Comment From: gopherbot

Change https://go.dev/cl/626615 mentions this issue: cmd/compile: allow more types for wasmimport/wasmexport parameters and results

Comment From: cherrymui

The proposal reads

*struct

If the struct has any fields, it must embed structs.HostLayout

This sounds like pointer to empty struct is allowed. I'm not sure this is useful, though, as I don't think the host can do anything with a pointer to an empty struct. Should we disallow it (by just removing the "If the struct has any fields" clause)? If one really wants a pointer to a struct with no (real) field, they can still do *struct { _ structs.HostLayout }.

Comment From: ydnar

The proposal reads

*struct

If the struct has any fields, it must embed structs.HostLayout

This sounds like pointer to empty struct is allowed. I'm not sure this is useful, though, as I don't think the host can do anything with a pointer to an empty struct. Should we disallow it (by just removing the "If the struct has any fields" clause)? If one really wants a pointer to a struct with no (real) field, they can still do *struct { _ structs.HostLayout }.

We actually use this quite a bit in the TinyGo implementation of wasip2. Given that it has zero width, therefore no fields to lay out, the exception seems reasonable.

Here’s the implementation in TinyGo, for what it’s worth: https://github.com/tinygo-org/tinygo/blob/2a76ceb7dd5ea5a834ec470b724882564d9681b3/compiler/symbol.go#L466-L491

Edit: where this is used in practice is not typically a pointer to an empty struct, but a struct that contains a field that is an empty struct, e.g. for interface conformance:

type S struct {
    _ structs.HostLayout
    embedded
    Name string
}

type embedded struct{}

func (embedded) isEmbedded() {}

Edit 2: I suppose we could replace all instances of struct{} in generated code with [0]T, but struct{} is more idiomatic Go.

Comment From: cherrymui

where this is used in practice is not typically a pointer to an empty struct, but a struct that contains a field that is an empty struct, e.g. for interface conformance

Fair point. Thanks for the example.

Comment From: gopherbot

Change https://go.dev/cl/627059 mentions this issue: runtime, syscall: use pointer types on wasmimport functions

Comment From: cherrymui

CL 626615 implemented this for wasm. This will be included in Go 1.24.

The wasm32 port will adopt this when the port lands. (Move the backlog milestone for wasm32.)

Comment From: dmitshur

@cherrymui Do you think that wasm change (in CL 626615) is something to be mentioned in Go 1.24 release notes?

Comment From: cherrymui

Yes, it should be mentioned. I'll do it along with other Wasm changes. Thanks.

Comment From: gopherbot

Change https://go.dev/cl/634036 mentions this issue: doc/next: add release notes for Wasm changes

Comment From: gopherbot

Change https://go.dev/cl/633776 mentions this issue: cmd/compile: document wasmexport directive, update permitted types for wasmimport

Comment From: gopherbot

Change https://go.dev/cl/638881 mentions this issue: cmd/compile: improve compiler directive docs

Comment From: gabyhelp

AI-generated issue overview

Go Wasm Import/Export Type Constraints Relaxation

This proposal suggests relaxing the type constraints for the go:wasmimport compiler directive, which allows importing functions from Wasm modules into Go. Currently, strict limitations exist on the types allowed as input and result parameters due to the mismatch between 64-bit and 32-bit architectures between Go and Wasm hosts. The arrival of the 32-bit Wasm port (issue) mitigates this issue. The proposal aims to broaden the set of permissible types for both wasm and wasm32 architectures. (@johanbrandhorst, issue)

The proposal defines a set of "small integer types" ([u]int[8|16]) and lists several types that would be allowed as input parameters: bool, int32, uint32, int64, uint64, float32, float64, string, uintptr, unsafe.Pointer, pointers to allowed types, pointers to structs (with restrictions based on the architecture and requirement to embed structs.HostLayout), and pointers to arrays of allowed types. All input parameter types except string would also be allowed as result parameter types. Several types, like channels, complex numbers, functions, interfaces, maps, and slices would remain disallowed. (@johanbrandhorst, issue)

The proposal also acknowledges that no compatibility guarantees can be provided across Go compiler versions due to Go's unspecified struct layout. Conversion rules between Go types and Wasm parameter types are outlined. The rationale for disallowing strings as result parameters is due to Wasm's limitation on result parameters. The proposal further discusses the challenges of supporting GOARCH=wasm, which uses 64-bit pointers, and explains why certain types like slices and maps remain disallowed. The proposal also touches upon future considerations, such as WASI Preview 2 and the potential for supporting structs and arrays by value. (@johanbrandhorst, issue)

Discussion Themes

Alignment and structs.HostLayout: Discussion centered on the alignment of 8-byte quantities within structs passed to wasm32. The consensus reached was to use 8-byte alignment for 64-bit types in structs passed to the Wasm host platform when they're marked with structs.HostLayout. This tag signifies host compatibility and is crucial for ensuring proper alignment. (@dr2chase, comment; @ydnar, comment; @evanphx, comment; @dr2chase, comment)
String Handling and ABI Specifications: Questions arose about the representation and handling of strings in Wasm/WASI and the exact ABI specifications for structs and arrays. For wasip1, strings are treated as a (*byte, int) tuple without encoding constraints. The ABI for structs and arrays involves flattening them into individual call parameters. (@cherrymui, comment; @johanbrandhorst, comment)
Passing Structs and Arrays by Value: Discussion arose on whether or not to support passing structs and arrays by value, which the Component Model Canonical ABI supports, while the Basic C ABI has different constraints on this. There was agreement to simplify the initial implementation by removing support for passing structs and arrays by value. This feature could be revisited later if needed. (@johanbrandhorst, comment; @aykevl, comment; @cherrymui, comment)
Pointer Passing and GC Safety: Concerns about pointer passing and GC safety, particularly when returning Go pointers from wasmimport functions or passing them to wasmexport functions were discussed. It was agreed to adopt conventions similar to CGO to ensure memory safety and proper GC behavior. This includes disallowing the host from writing Go pointers to Go memory and requiring pinning of Go pointers held by the host. (@ydnar, comment; @johanbrandhorst, comment; @cherrymui, comment)
Allowed Types for GOARCH=wasm: Discussion also covered the allowed types for GOARCH=wasm, which differs from wasm32 in integer and pointer sizes. The proposal aims to provide some relaxations for wasm as well but with more restrictions due to the 64-bit pointer size mismatch with the host. (@cherrymui, comment; @johanbrandhorst, comment)
int and uint Types: Concerns were raised about allowing int and uint due to their platform-dependent sizes. The proposal was updated to disallow these types to avoid ambiguity. Similar concerns about uint8 and uint16 led to their removal as allowed parameter types but they are still permitted as pointer types or struct fields. (@aclements, comment; @ydnar, comment; @johanbrandhorst, comment)

Resolution

The proposal was accepted. The final version clarifies allowed types, string handling, pointer passing conventions, and the differences in support between wasm and wasm32 architectures. The implementation for wasm was completed in Go 1.24 (comment), while the wasm32 implementation will follow upon the port's completion.

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

Comment From: dsseng

All the mentioning changes have been merged. @johanbrandhorst could this issue be already resolved?

Comment From: johanbrandhorst

Yep, I think we can close this!

Golang cmd/compile: relax wasm/wasm32 function import signature type constraints

Background

Proposal

Discussion

Compatibility guarantees

Type conversion rules

Strings

Supporting GOARCH=wasm

Supporting [u]int, [u]int8, [u]int16 as concrete parameters

Supporting slices, maps

Related proposals

struct.Hostlayout

66408 proposes a way for users to request that struct layout is host compatible. Our proposal depends on the definitions put forward in this proposal for struct parameters.

Future work

WASI Preview 2 (AKA WASI 0.2)

Supporting struct and [...]T by value

Contributors

AI-generated issue overview

Go Wasm Import/Export Type Constraints Relaxation

Discussion Themes

Resolution

Supporting `GOARCH=wasm`

Supporting `[u]int`, `[u]int8`, `[u]int16` as concrete parameters

66408 proposes a way for users to request that struct layout is host compatible. Our proposal depends on the definitions put forward in this proposal for `struct` parameters.

Supporting `struct` and `[...]T` by value