Proposal Details

Go 1.22 has provided http.ServeMux with supporting of pattern match for requests. While I'm using it, I tried to get the matched pattern of that request. With help of *ServeMux.Handler, I can successfully get matched handler and pattern, but later when I run debug of the code, I found in *http.Request itself, there is a private pat which is exactly matched pattern info. I suggest we export the pat for public access or provide some function to allow user to get the matched pattern, so that we can avoid much logic to use *ServeMux.Handler to parse the request again. This reduces huge resource usage in high load services.

The matched pattern info is usually very useful when we do analysis for requests. We can group requests by their pattern and make the analysis more generic. This is a very high frequency usage scenario which makes reducing the resource usage very worthy.

sample code:

func TestServeMux(t *testing.T) {
    m := http.NewServeMux()
    m.Handle("GET /{name}", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        t.Log(r.PathValue("name"))
    }))
    http.ListenAndServe(":8999", m)
}

When we do request GET http://127.0.0.1:8999/aloha, we can see such info in debug console.

Golang net/http: expose matched pattern in Request

Update 2024/03/28: As we talked in comments, @jba 's suggestion is a good solution https://github.com/golang/go/issues/66405#issuecomment-2023331350

// Pattern returns the pattern that matched this request, if the request
// resulted from matching a pattern registered on a ServeMux.
// Otherwise, it returns the empty string.
func (*Request) Pattern() string

Update 2024/04/11: After many discussions, the latest proposal can be found here https://github.com/golang/go/issues/66405#issuecomment-2048393929 In short, add a Pattern field in http.Request to expose matched pattern info and can be set by third party routers.

Update 2024/05/13: Final design: https://github.com/golang/go/issues/66405#issuecomment-2101500192 The proposal is to add a Pattern string field to net/http.Request. ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux. In Go 1.21 compatibility mode the field is unset. The field is otherwise left untouched. Third-party routers may set it if they wish.

Comment From: seankhliao

see also #61551

Comment From: sillydong

Similar to #61551 but instead of setting pattern info in Context, I suggest we export it directly in http.Request where it already has that info and the pattern is actually corresponding to request itself instead of any Context.

Comment From: AlexanderYastrebov

Maybe the string pattern value could be made accessible via special name r.PathValue("*") (note also #64910)

Comment From: sillydong

We definitely need a way to get pattern info in lower cost instead of using *ServeMux.Handler to parse request again.

For now there are three suggestions for getting that

  • [x] expose pat property in request directly or provide a function Pattern to get pattern string
  • [ ] let servemux set matched pattern info in context, people get pattern from context by fixed key https://github.com/golang/go/issues/61551
  • [ ] use a special name r.PathValue("*") to get pattern

Comment From: ianlancetaylor

CC @neild @jba

Comment From: jba

I wouldn't be opposed to adding:

// Pattern returns the pattern that matched this request, if the request
// resulted from matching a pattern registered on a ServeMux.
// Otherwise, it returns the empty string.
func (*Request) Pattern() string

Comment From: jba

@sillydong, if you're OK with my suggestion above, please edit the top post to refer to my comment. You may also want to edit the issue title.

Comment From: DeedleFake

What about custom mux implementations? Should they be able to store whatever pattern they used in the Request?

Comment From: neild

Request.PathValue already conveys information from the mux, without any way for custom mux implementations to set it.

Perhaps there should be some way for custom muxes to set Request.Pattern and Request.PathValue, but I think that can be a separate proposal. Adding Pattern doesn't seem to make things any worse for custom muxes than they are now.

Comment From: seankhliao

Request.PathValue already conveys information from the mux, without any way for custom mux implementations to set it.

it can be set with Request.SetPathValue

Comment From: neild

it can be set with Request.SetPathValue

Today I learned!

In that case, I agree that there should be a corresponding SetPattern, or Pattern should just be a field of Request.

Comment From: jba

I'm OK with a Pattern field.

Comment From: sillydong

To expose the Pattern field, we have to change private pattern to public Pattern, also the fields in pattern like str,method,host,segments,loc should be public, and this makes type segment should also be public. This is making things complicate. I suggest we provide Pattern()string and SetPattern(string) functions. In SetPattern(string) we can call existing parsePattern to parse the parameter and set to http.Request.pat. This can be much easier and cleaner. @jba @neild what do you think?

Comment From: gopherbot

Change https://go.dev/cl/574997 mentions this issue: net/http: add Pattern and SetPattern in Request to return matched pattern info

Comment From: neild

To expose the Pattern field, we have to change private pattern to public Pattern, also the fields in pattern like str,method,host,segments,loc should be public, and this makes type segment should also be public.

Or we could just have:

type Request {
  // Pattern is the [ServeMux] pattern that matched the request, if any.
  Pattern string
}

That seems pretty simple.

If we do have a SetPattern, then we need to consider how to handle a pattern that doesn't parse. What if the user is using a mux with patterns that don't correspond to the ones http.ServeMux uses? That seems likely; I'd expect the entire reason someone would use an alternate mux is to get different pattern matching features.

Comment From: sillydong

@neild No. Pattern is actually a structured data. It contains segments and pattern string itself. Segments are used while getting path value. If we turn pattern into string, we lost everything.

I have created PR for Pattern()string and SetPattern(string)error, you can check it out.

Comment From: jba

@sillydong I think you're missing @neild's point. Your code for SetPattern parses the pattern according to http.ServeMux's rules, but someone else's pattern syntax may be entirely different. Since we can't know how other patterns work, the best we can do is use a string.

Comment From: sillydong

I think both you guys explained makes sense. But the problem is if we provide a function to set a string pattern, we have to create a new field to save that string info. That makes us not able to take advantage of Requst.pat. And also the Pattern function would have to return that string value instead of the real pattern string in Request.pat.str. This is kind of weird. I'm not quite sure about if this is a good way to handle the pattern info.

Comment From: jba

We would be duplicating information, that's true. Request.Pattern would be set from Request.pat.str. If we were just supporting retrieving the pattern, I probably would have a Pattern() string method instead that returns Request.pat.str. But since we are allowing setting it, and setting it to an arbitrary string that might not be a valid pattern, having a separate field makes more sense.

Comment From: sillydong

I thought about all possible solutions, if we go with ability of allowing third-party frameworks set pattern, we have to provide a simple extra field to save it. And if it's empty, we still return Request.pat.str.

Comment From: AlexanderYastrebov

I thought about all possible solutions, if we go with ability of allowing third-party frameworks set pattern, we have to provide a simple extra field to save it. And if it's empty, we still return Request.pat.str.

Maybe do

    } else {
        h, _, r.pat, r.matches = mux.findHandler(r)
        if r.pat != nil {
            r.Pattern = r.pat.str
        }
    }

around here https://github.com/golang/go/blob/20f052c83c380b8b3700b7aca93017178a692d78/src/net/http/server.go#L2684-L2686 to avoid adding methods.

Comment From: sillydong

@AlexanderYastrebov coorect, a single string field to save the pattern string. I will update the PR in this way.

Comment From: jba

A ServeMux can itself be used as a handler. What should the value of r.Pattern be in this case:

mux := http.NewServeMux()
http.Handle("/foo", mux)
mux.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) { /* r.Pattern? */})

Comment From: sillydong

When I first see this code, I would say the r.Pattern should be /bar since it's inside mux, but when I wrote this code and run ListenAndServe, I found it always returns 404, no matter I request /foo or /foo/bar. The reason is ServeMux cannot match the right pattern. If we run with the following code, we can request with /foo successfully and r.Pattern is /

mux := http.NewServeMux()
http.Handle("/foo", mux)
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { /* r.Pattern? */})

Comment From: seankhliao

I believe the correct snippet should be

http.Handle("/foo/", mux)

Comment From: rsc

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

Comment From: jba

@sillydong My code was typed without testing, thanks for fixing it. I agree that it should be the pattern of the innermost ServeMux. That's what Request.PathValue does, and it should be consistent.

Can I ask you once again to change the top post to reflect the current proposal, which is to have a Pattern field instead of a method.

Are there any unresolved objections to that?

Comment From: rsc

FWIW, it is usually clearer to post the updated proposal in a new message, and then update the first comment to just say

"The latest proposal is \."

than to rewrite history. Rewriting history is confusing to anyone trying to get up to speed on the conversation.

Comment From: jba

This comment describes the current proposal.

Add a Pattern field of type string to net/http.Request. ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler. The field is otherwise left untouched. Third-party routers may set it if they wish.

Comment From: sillydong

Instead of checking r.pat, I took usage of the returned string of mux.findHandler(r). Easier and cleaner.

@rsc Proposal is updated and code change has been done. Please help push forward the progress. Thanks a lot. PR: golang.org/cl/574997

Comment From: rsc

Have all remaining concerns about this proposal been addressed?

The proposal is to add a Pattern string field to net/http.Request. ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux. In Go 1.21 compatibility mode the field is unset. The field is otherwise left untouched. Third-party routers may set it if they wish.

Comment From: sillydong

In Go 1.21 compatibility mode the field is unset.

For this part, I think we can set matched pattern string to Pattern field because in Go 1.21 the mux is also able to run finding matched handler, it also returns matched pattern string. Golang net/http: expose matched pattern in Request

Comment From: seankhliao

Does accepting this mean declining #61551 ? And will http.StripPrefix modify Request.Pattern ?

Comment From: jba

Does accepting this mean declining https://github.com/golang/go/issues/61551 ?

I think yes. I don't believe that proposal adds anything once we have Request.Pattern.

And will http.StripPrefix modify Request.Pattern ?

No. I don't even know what that would mean in some cases. For example, say the pattern is /{a}, the path is /foo, and the argument to StripPrefix is /f. That works today and changes the path to oo (it literally calls strings.TrimPrefix). What would it do to the pattern?

Comment From: seankhliao

So there's #64909 to teach StripPrefix about path patterns.

I was wondering about a more limited form where the pattern is processed the same as Request.URL.Path and RawPath now, so a literal match would be updated (request /a/b, handle /a/ stripprefix /a/, pattern /b), but a pattern match wouldn't (request /a/b, handle /{x}/, stripprefix ???, pattern /{x}/).

Comment From: sillydong

In Go 1.21 compatibility mode the field is unset.

For this part, I think we can set matched pattern string to Pattern field because in Go 1.21 the mux is also able to run finding matched handler, it also returns matched pattern string.

@rsc @jba Hi, what do you think of this update? I think it's fair enough to also update the pattern for mux121 since it does have the pattern info from findHandler, we should not ignore it.

Comment From: sillydong

Hi @rsc @jba , is there any update for this proposal? What do you think of also provide pattern info for Go 1.21 mux?

Comment From: rsc

We were away last week. This issue will move to likely accept this week.

We should leave the old mux behavior unchanged. It is still accessible from a GODEBUG but we want people to move off it. We are not updating it, and eventually it may be removed. Adding new API to it gives the opposite impression.

Comment From: rsc

Based on the discussion above, this proposal seems like a likely accept. — rsc for the proposal review group

The proposal is to add a Pattern string field to net/http.Request. ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux. In Go 1.21 compatibility mode the field is unset. The field is otherwise left untouched. Third-party routers may set it if they wish.

Comment From: rsc

No change in consensus, so accepted. 🎉 This issue now tracks the work of implementing the proposal. — rsc for the proposal review group

The proposal is to add a Pattern string field to net/http.Request. ServeMux.ServeHTTP will set the field to the pattern of the matching handler before dispatching to that handler when using the new Go 1.22 HTTP mux. In Go 1.21 compatibility mode the field is unset. The field is otherwise left untouched. Third-party routers may set it if they wish.