The following Go program reports that the function makes an allocation to do the string to []byte conversion, even though the compiler reports the conversion as non-escaping.
https://play.golang.org/p/y2d2RiYqK5
It prints:
48 B/op 1 allocs/op
Given that the compiler has marked it as non-escaping, I'd expect no allocations.
go version devel +eab99a8 Mon Jun 26 21:12:22 2017 +0000 linux/amd64
Comment From: randall77
It's non-escaping, but its size is unknown. We need to allocate []byte of unknown size on the heap.
Comment From: bcmills
We shouldn't even need to do that if the slice isn't mutated, right? (How hard would it be to make the escape analysis also do mutability?)
Comment From: rogpeppe
For the record I encountered this issue when passing a string to hex.Decode. Mutability analysis would be amazing, but just avoiding the allocation would be nice.
Comment From: bradfitz
Dup of #2205 ?
Comment From: rogpeppe
Although the particular example would be fixed if #2205 was fixed, I don't think it's quite the same issue. I hadn't realised that all variable-length allocations escaped (strictly speaking that's not true, because []byte(string) where len(string)<=32 is stack-allocated), and I'm not sure that others do either.
I'm probably not the only one who was using -gcflags -m output as a proxy for whether there's an allocation or not. Given that we're (I presume) not about to make some kind of instant-free arena for variable-length items, perhaps another way of addressing this issue might be to change the output printed by the -m flag to make it clearer that this the result of this kind of operation may actually escape to the heap.
Comment From: robpike
Sure, it's escaped to the heap but that's as far as it will get. The guard towers that surround the heap are well-manned and your memory will never get outside the heap's walls. You can relax.
Comment From: andybons
@rogpeppe is your proposal to change the output of the -m flag to make this clearer? Is this worthy of a proposal or NeedsDecision? Trying to figure out next steps.
Comment From: bradfitz
Unplanned, needs fix, performance.
No decision needed. This is just work, and somewhat tricky work.
Comment From: andybons
Thanks, Brad. Updated.
Comment From: josharian
Given that we're (I presume) not about to make some kind of instant-free arena for variable-length items
Cross-reference: https://github.com/golang/go/issues/20533
Comment From: philjb
Should this ticket remain open now that https://github.com/golang/go/issues/2205 is resolved? The original demonstration code reports zero allocations now (as expected with 2205 fix).
Comment From: randall77
Yes, I think this is fixed.
Comment From: thepudds
I will briefly add that @rogpeppe in https://github.com/golang/go/issues/20881#issuecomment-312567987 seems to have said he thought #2205 was not exactly the same as this issue here:
Although the particular example would be fixed if https://github.com/golang/go/issues/2205 was fixed, I don't think it's quite the same issue.
If I followed, it seems part of the concern might have been whether it's clear what is happening from the -gcflags=-m logs.
FWIW, I think there is a reasonably clear log emitted with the -gcflags=-m flag for the new-ish zero-copy conversion for this example (thanks to the work done in #2205), though that's based on a quick look and maybe I missed something:
t.go:23:14: zero-copy string->[]byte conversion
t.go:22:13: s does not escape
t.go:23:14: ([]byte)(s) does not escape
In any event, there might be opportunities to improve the logging, but that might now be better tracked elsewhere (and/or maybe with a different example than the one at the top of this issue).