Proposal Details
Currently slog package a logger only support one handler. For most language like python, A log module should be sent to multiple targets for output simultaneously for a logger. This feature is necessary， I propose adding multiple handlers support for logger. cc @golang/proposal-review
Comment From: seankhliao
it's unclear why this needs to be in the standard library when it can accomplished as a handler. as the first example I found: https://github.com/samber/slog-multi
Comment From: lxl-renren
I agree other third-party library like "slog-multi" accomplished this functionality. Although third-party solutions exist, standardizing this functionality offers several benefits: - Consistency and Ease of Use: A standardized approach ensures consistent behavior and API across different projects using the slog package. This reduces the learning curve and simplifies adoption for new users. - Discoverability and Integration: Integrating with a standard library feature is often easier. Users don't need to search for and manage dependencies on external packages, making development and maintenance smoother. - Quality and Maintenance: Standard library features are actively maintained by the Go team, receiving regular updates, bug fixes, and security patches. This ensures a reliable and well-supported solution for users.
Using third-party library like "slog-multi" may not be maintained consistently, potentially leading to compatibility issues in the future.
Comment From: ianlancetaylor
CC @jba
Comment From: jba
@lxl-renren While I agree with you about the values of standardization, I don't think this particular feature warrants it. It is easy enough to write and not needed often enough to justify the added maintenance burden to the Go team.
However, that is just my opinion, and we can and should let the proposal process take its course.
Comment From: fishjam
I want this feature too. As I know, lots of project will save logs to multi target , example: - local file's log with more detail(level is debug); - remote elk log with less detail ( example: level is info ) - and maybe alarm email log ( example: level is error )
current slog can not support such case.
Comment From: alireza-fa
If we need to log in two different outputs, we can use io.Multiwriter logger = slog.New( slog.NewJSONHandler(io.MultiWriter(fileWriter, os.Stdout), &slog.HandlerOptions{}), ) If we need to log in two different output with different values, example: - local file's log with more detail(level is debug); - remote elk log with less detail ( example: level is info ) - and maybe alarm email log ( example: level is error )
we can warp slog in a custom Logger and handling this sutiations.
Comment From: featuriz
io.Multiwriter works. But it fixed logLevel for all.
@alireza-fa can you please provide some code to start with
Comment From: jba
@featuriz here is a sketch of the code:
The MultiHandler would be a struct that holds a slice of Handlers.
Its Enabled method would return true if any of the contained handlers' Enabled methods returns true.
Its Handle method would re-check the Enabled method of each contained handler, and if true, it would call Handle on that handler.
The WithGroup and WithAttrs methods would call those methods on every contained handler, replacing each handler with the method's return value.
Comment From: ianthehat
Its Handle method would re-check the Enabled method of each contained handler, and if true, it would call Handle on that handler.
I am not sure it needs to do this, the point of Enabled was just an optimization to avoid building the message, by the time you call the multi Handle that optimization no longer applies and I don't think there is any guarantee in the system that Handle does not get called just because Enabled is false, so it should just unconditionally deliver the message to all the handlers and let them reject ones they did not want.
I have written this code a couple of times for bifurcation, not bothered to generalize it to N handlers though.
Comment From: alireza-fa
Yes, I will definitely provide you with a practical example:
package logger
import (
"io"
"log/slog"
"os"
"gopkg.in/natefinch/lumberjack.v2"
)
const (
defaultFilePath = "logs/logs.json"
defaultUserLocalTime = false
defaultFileMaxSizeInMB = 10
defaultFileAgeInDays = 30
defaultLogLevel = slog.LevelInfo
)
type Config struct {
FilePath string `koanf:"file_path"`
UseLocalTime bool `koanf:"use_local_time"`
FileMaxSizeInMB int `koanf:"file_max_size_in_mb"`
FileMaxAgeInDays int `koanf:"file_max_age_in_days"`
LogLevel int `koanf:"log_level"`
}
var l *slog.Logger
func init() {
fileWriter := &lumberjack.Logger{
Filename: defaultFilePath,
LocalTime: defaultUserLocalTime,
MaxSize: defaultFileMaxSizeInMB,
MaxAge: defaultFileAgeInDays,
}
l = slog.New(slog.NewJSONHandler(io.MultiWriter(fileWriter, os.Stdout), &slog.HandlerOptions{
Level: defaultLogLevel,
}))
}
func L() *slog.Logger {
return l
}
func New(cfg Config, opt *slog.HandlerOptions, writeInConsole bool) *slog.Logger {
fileWriter := &lumberjack.Logger{
Filename: cfg.FilePath,
LocalTime: cfg.UseLocalTime,
MaxSize: cfg.FileMaxSizeInMB,
MaxAge: cfg.FileMaxAgeInDays,
}
if writeInConsole {
return slog.New(slog.NewJSONHandler(io.MultiWriter(fileWriter, os.Stdout), opt))
}
return slog.New(slog.NewJSONHandler(fileWriter, opt))
}
In this example, we specified that the logs should be displayed in the console as well as written to the specified file.
@featuriz
Comment From: jba
I don't think there is any guarantee in the system that Handle does not get called just because Enabled is false
The slog package will not call Handle if Enabled returns false, so it is still worth checking if none of the contained handlers is enabled.
Of course, slog.Handler.Handle is a public method, so anyone can call it whenever they want, but I don't think that's what you meant.
Comment From: featuriz
@alireza-fa you got misunderstood. I have mentioned that io.MultiWriter works, means that, I have implement already. I need the further things that you have mentioned in your comments:
If we need to log in two different output with different values, example:
- local file's log with more detail(level is debug);
- remote elk log with less detail ( example: level is info )
- and maybe alarm email log ( example: level is error )
we can warp slog in a custom Logger and handling this sutiations.
- Save in REST API Server (saving outside)
- By using (io.MultiWriter) I came up with a problem. That is levels
- on development (I can do with an environment bool check), in console, I need all logs , (it can be set by option level)
- system: 2 files (all,error) to be saved. One with all the log and another with level error (error and warning)
- API: save as per level (because, some logging server are expensive- so only error logging is enough.
io.MultiWriter: it is limited. We can save same log level in different places.
Goal: Different places, each with it's own level.
Important: set as defined log
Comment From: featuriz
Thanks to @jba for the sketch and I saw your youtube video about slog. This helped me to create my own MultiHandler.
@seankhliao has mentioned in 1st comment about this. Instead of reinventing, I go for slog-multi. It also has grafana loki, which I use.
Comment From: FotiadisM
With the addition of slog to the standard library, some log instrumentation libraries have taken the approach of implementing a slog.Handler.
For example, Open Telemetry has created otelslog, and as far as I understand using it is the recommended approach (instead of collecting the logs from a file, stdout, container, etc)
It seems to me that every application that needs to add any sort of log instrumentation will have to implement a multi-handler of some sort.
I think this would be a small, welcomed change to the standard library.
Comment From: prochac
I came here just because of the same thing: OTel slog.
it's unclear why this needs to be in the standard library when it can accomplished as a handler. as the first example I found: https://github.com/samber/slog-multi
Just because something can be implemented outside stdlib doesn't mean the Go stdlib couldn't be helpful. Structured logging could and was created outside stdlib, so why it has been added? The same with http router, etc.
This is how WebSockets can be implemented outside stdlib, every few years new recomended module: https://cs.opensource.google/go/x/net/+/97edce0b2e423f6a8debb459af47f4a3cb4ff954 https://cs.opensource.google/go/x/net/+/1a5e07d1ff72da30f660d3c95b312a20be31173c https://cs.opensource.google/go/x/net/+/ad92d3db360ba0127e5d4d17cc94b2bb4c09e168 https://cs.opensource.google/go/x/net/+/f88258d67e0f0f144c79964ca05bb81d51ee8411
Comment From: jba
The sentiment and the emoji voting seem to be in favor of this. I'm now in favor.
The arguments for it are:
- Although it is straightforward to implement, it is not trivial. There are a few pitfalls, especially if you've never written a handler.
- Its implementation will provide another example of how to implement a handler.
- Having it in the standard library will reduce dependencies.
- There is precedent for multiplexing implementations of common interfaces: io.MultiReader and io.MultiWriter.
- There seems to be some desire for it.
So, I propose we add the following to the log/slog package:
// MultiHandler returns a handler that invokes all the given Handlers.
// Its Enable method reports whether any of the handlers' Enabled methods return true.
// Its Handle, WithAttr and WithGroup methods call the corresponding method on each of the enabled handlers.
func MultiHandler(handlers ...Handler) Handler
/cc @aclements
Comment From: nstandif
@jba A few thoughts: - How would this interface isolate glacial and flaky handlers from supported ones? - Would they block or be asynchronous? Why? - Would they all fail or only some fail? If they all fail, what would happen?
Comment From: jba
Good questions.
- How would this interface isolate glacial and flaky handlers from supported ones?
It wouldn't attempt to. Those problems exist even for a single handler. If you were really worried about such things you'd build a Handler that wrapped a single handler and did the best it could, like having a timeout and using recover.
- Would they block or be asynchronous? Why?
Handler methods would be called sequentially, as with io.MultiWriter. Following its documentation, I would add the phrase "one at a time".
As above, whether to be synchronous or not should be the decision of a single handler, and you can write a wrapper for a Handler that makes it asynchronous if you want. Admittedly you'd need to write your own version of a mult-handler if you wanted to wait for all the sub-handlers to finish, but unless there's a strong desire for that I'd rather avoid the complexity.
- Would they all fail or only some fail? If they all fail, what would happen?
The MultiHandler.Handle method would return the errors.Join of the individual Handle methods. In other words, every handler would get a chance to run.
Comment From: nstandif
The MultiHandler.Handle method would return the errors.Join of the individual Handle methods. In other words, every handler would get a chance to run.
This behavior diverges a bit from the io.MultiWriter implementation: "If a listed writer returns an error, that overall write operation stops and returns the error; it does not continue down the list." Looking at the source https://cs.opensource.google/go/go/+/refs/tags/go1.24.1:src/io/multi.go;l=83-95 confirms this.
The advantage of following the MultiWriter implementation is that it provides some continuity between the two modules, so I don't have to think about module x behaving differently from module y on multiple writes. Also, short circuiting is generally easier to debug.
As above, whether to be synchronous or not should be the decision of a single handler, and you can write a wrapper for a Handler that makes it asynchronous if you want. This is the right approach and I think it could even be extended to what I described above if you don't want them to short circuit.
On Mon, Mar 17, 2025 at 10:47 AM Jonathan Amsterdam ***@***.*** wrote:
Good questions.
- How would this interface isolate glacial and flaky handlers from supported ones?
It wouldn't attempt to. Those problems exist even for a single handler. If you were really worried about such things you'd build a Handler that wrapped a single handler and did the best it could, like having a timeout and using recover.
- Would they block or be asynchronous? Why?
Handler methods would be called sequentially, as with io.MultiWriter. Following its documentation https://pkg.go.dev/io#MultiWriter, I would add the phrase "one at a time".
As above, whether to be synchronous or not should be the decision of a single handler, and you can write a wrapper for a Handler that makes it asynchronous if you want. Admittedly you'd need to write your own version of a mult-handler if you wanted to wait for all the sub-handlers to finish, but unless there's a strong desire for that I'd rather avoid the complexity.
- Would they all fail or only some fail? If they all fail, what would happen?
The MultiHandler.Handle method would return the errors.Join of the individual Handle methods. In other words, every handler would get a chance to run.
— Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/65954#issuecomment-2730380909, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH6MGJE33POLMNFDNPAKST2U4DBXAVCNFSM6AAAAABD3Z7MUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQGM4DAOJQHE . You are receiving this because you are subscribed to this thread.Message ID: @.*> [image: jba]jba left a comment (golang/go#65954) https://github.com/golang/go/issues/65954#issuecomment-2730380909
Good questions.
- How would this interface isolate glacial and flaky handlers from supported ones?
It wouldn't attempt to. Those problems exist even for a single handler. If you were really worried about such things you'd build a Handler that wrapped a single handler and did the best it could, like having a timeout and using recover.
- Would they block or be asynchronous? Why?
Handler methods would be called sequentially, as with io.MultiWriter. Following its documentation https://pkg.go.dev/io#MultiWriter, I would add the phrase "one at a time".
As above, whether to be synchronous or not should be the decision of a single handler, and you can write a wrapper for a Handler that makes it asynchronous if you want. Admittedly you'd need to write your own version of a mult-handler if you wanted to wait for all the sub-handlers to finish, but unless there's a strong desire for that I'd rather avoid the complexity.
- Would they all fail or only some fail? If they all fail, what would happen?
The MultiHandler.Handle method would return the errors.Join of the individual Handle methods. In other words, every handler would get a chance to run.
— Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/65954#issuecomment-2730380909, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH6MGJE33POLMNFDNPAKST2U4DBXAVCNFSM6AAAAABD3Z7MUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZQGM4DAOJQHE . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Comment From: jba
This behavior diverges a bit from the io.MultiWriter implementation
True, but for logging I think it makes more sense. In fact, if you're logging to two handlers, there's a good chance that one is a fallback if the other fails (for example, a remote logging service and standard error). So continuing after an error makes more sense.
Comment From: olekukonko
I also support a proposal for enhancing the slog package functionality by enabling it to accommodate multiple handlers, allowing logs to be processed in parallel. Here's a simple demonstration of a MultiHandler that integrates several existing handlers:
// simple entry
type LogEntry struct {
UID string
Timestamp time.Time
Message string
Level slog.Level
Attrs map[string]interface{}
}
// MultiHandler combines multiple handlers
type MultiHandler struct {
handlers []slog.Handler
queue chan LogEntry
wg sync.WaitGroup
}
func NewMultiHandler(handlers ...slog.Handler) *MultiHandler {
mh := &MultiHandler{
handlers: handlers,
queue: make(chan LogEntry, 1000),
}
mh.startWorkers()
return mh
}
func (mh *MultiHandler) startWorkers() {
mh.wg.Add(1)
go func() {
defer mh.wg.Done()
batch := make([]LogEntry, 0, 10)
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
for {
select {
case entry, ok := <-mh.queue:
if !ok {
mh.flushBatch(batch)
return
}
batch = append(batch, entry)
if len(batch) >= 10 {
mh.flushBatch(batch)
batch = batch[:0]
}
case <-ticker.C:
if len(batch) > 0 {
mh.flushBatch(batch)
batch = batch[:0]
}
}
}
}()
}
func (mh *MultiHandler) flushBatch(batch []LogEntry) {
for _, h := range mh.handlers {
for _, entry := range batch {
r := slog.NewRecord(entry.Timestamp, entry.Level, entry.Message, 0)
for k, v := range entry.Attrs {
r.AddAttrs(slog.Any(k, v))
}
if err := h.Handle(context.Background(), r); err != nil {
fmt.Fprintf(os.Stderr, "Handler error: %v\n", err)
}
}
}
}
func (mh *MultiHandler) Handle(ctx context.Context, r slog.Record) error {
attrs := make(map[string]interface{})
r.Attrs(func(a slog.Attr) bool {
attrs[a.Key] = a.Value.Any()
return true
})
mh.queue <- LogEntry{
UID: ulid.Make().String(),
Timestamp: r.Time,
Message: r.Message,
Level: r.Level,
Attrs: attrs,
}
return nil
}
func (mh *MultiHandler) Enabled(ctx context.Context, level slog.Level) bool {
return true // Accept all levels by default
}
func (mh *MultiHandler) WithAttrs(attrs []slog.Attr) slog.Handler {
newHandlers := make([]slog.Handler, len(mh.handlers))
for i, h := range mh.handlers {
newHandlers[i] = h.WithAttrs(attrs)
}
return NewMultiHandler(newHandlers...)
}
func (mh *MultiHandler) WithGroup(name string) slog.Handler {
newHandlers := make([]slog.Handler, len(mh.handlers))
for i, h := range mh.handlers {
newHandlers[i] = h.WithGroup(name)
}
return NewMultiHandler(newHandlers...)
}
func (mh *MultiHandler) Close() error {
close(mh.queue)
mh.wg.Wait()
return nil
}
simple usage
// Combine handlers
multiHandler := NewMultiHandler(textHandler, clickHouseHandler, postgresHandler, lokiHandler)
logger := slog.New(multiHandler)
logger.Info("starting application", slog.String("version", "1.0"))
Comment From: jba
That's nice. But there are so many degrees of freedom in a concurrent implementation that it's unclear what should be hardcoded and what provided as options, or if there are other designs that might be better in some cases. I don't think we have the experience to bake one of these into the standard library. A sequential implementation is much more straightforward.
Comment From: nstandif
Not sure how idiomatic this approach is, but perhaps add a strategy pattern as the first parameter?
Like: NewMultiHandler(s MultiHandlerStrategy, h … Handlers)
Stdlib provides a reasonable default and others may extend for more functionality, like concurrency etc. On Tue, Mar 18, 2025 at 4:46 AM Jonathan Amsterdam @.***> wrote:
That's nice. But there are so many degrees of freedom in a concurrent implementation that it's unclear what should be hardcoded and what provided as options, or if there are other designs that might be better in some cases. I don't think we have the experience to bake one of these into the standard library. A sequential implementation is much more straightforward.
— Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/65954#issuecomment-2732914629, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH6MGJTWMCFO3JTN5CVVVD2VABQZAVCNFSM6AAAAABD3Z7MUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZSHEYTINRSHE . You are receiving this because you are subscribed to this thread.Message ID: @.*> [image: jba]jba left a comment (golang/go#65954) https://github.com/golang/go/issues/65954#issuecomment-2732914629
That's nice. But there are so many degrees of freedom in a concurrent implementation that it's unclear what should be hardcoded and what provided as options, or if there are other designs that might be better in some cases. I don't think we have the experience to bake one of these into the standard library. A sequential implementation is much more straightforward.
— Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/65954#issuecomment-2732914629, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH6MGJTWMCFO3JTN5CVVVD2VABQZAVCNFSM6AAAAABD3Z7MUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMZSHEYTINRSHE . You are receiving this because you are subscribed to this thread.Message ID: @.***>
Comment From: olekukonko
That's nice. But there are so many degrees of freedom in a concurrent implementation that it's unclear what should be hardcoded and what provided as options, or if there are other designs that might be better in some cases. I don't think we have the experience to bake one of these into the standard library. A sequential implementation is much more straightforward.
@jba I agree with you. This was just a quick proof of concept and didn’t fully account for all possible scenarios.
Not sure how idiomatic this approach is, but perhaps add a strategy pattern as the first parameter?
Like: NewMultiHandler(s MultiHandlerStrategy, h … Handlers)
Stdlib provides a reasonable default and others may extend for more functionality, like concurrency etc. …
@nstandif I agree that using a strategy pattern could be a good direction to explore. Based on the feedback here, I’ve made some updates. Check out what I came up with here: https://github.com/olekukonko/slogmulti. We can continue iterating on this until we feel it might be ready for a formal proposal.
Comment From: jba
That package looks fine, but it is another highly flexible, configurable type on top of one that is already flexible and configurable (slog.Logger). I'm hesitant to add so much additional complexity.
The best way to argue that something like this is worth including in the standard library is to show that fulfills a need in the community. A good recent example of that is #66365, where Sean was able to point to many cases where the new method would be useful. But the bar is higher here, because the amount of new behavior is higher.
Comment From: prochac
What async in this context means? That the handling is concurrent and we wait for all results. Or that the Handle spawns goroutine(s) and we don't wait for Handle error?
Comment From: FiloSottile
As a data point on need, I have now copied the following code in three different projects, which is all the projects I have adopted slog in.
type multiHandler []slog.Handler
// MultiHandler returns a Handler that handles each record with all the given
// handlers.
func MultiHandler(handlers ...slog.Handler) slog.Handler {
return multiHandler(handlers)
}
func (h multiHandler) Enabled(ctx context.Context, l slog.Level) bool {
for i := range h {
if h[i].Enabled(ctx, l) {
return true
}
}
return false
}
func (h multiHandler) Handle(ctx context.Context, r slog.Record) error {
var errs []error
for i := range h {
if h[i].Enabled(ctx, r.Level) {
if err := h[i].Handle(ctx, r.Clone()); err != nil {
errs = append(errs, err)
}
}
}
return errors.Join(errs...)
}
func (h multiHandler) WithAttrs(attrs []slog.Attr) slog.Handler {
handlers := make([]slog.Handler, 0, len(h))
for i := range h {
handlers = append(handlers, h[i].WithAttrs(attrs))
}
return multiHandler(handlers)
}
func (h multiHandler) WithGroup(name string) slog.Handler {
handlers := make([]slog.Handler, 0, len(h))
for i := range h {
handlers = append(handlers, h[i].WithGroup(name))
}
return multiHandler(handlers)
}
Comment From: jba
Thanks, @FiloSottile. Upvote the top post too if you haven't. Why copy, though? Is there some reason you can't put the code in a package that all the projects share?
Comment From: FiloSottile
Why copy, though? Is there some reason you can't put the code in a package that all the projects share?
It's just not enough code to justify a dependency.
Comment From: apparentlymart
I have added a 👍 upvote but I want to be explicit that what I am intending that to represent is a vote for a simple implementation with exactly the functionality shown in @FiloSottile's https://github.com/golang/go/issues/65954#issuecomment-2786268756 -- just a straightforward fanout of everything to all of the wrapped handlers -- and not trying to do anything more clever like queuing writes and processing things asynchronously.
I can imagine situations where more complex treatment like that would be helpful, but that introduces considerably more degrees of freedom in the design and implementation, meaning that the result would be considerably more code and thus higher maintenance burden. Third-party libraries are better positioned to handle the more advanced cases, particularly when those cases involve configurable policy like how often to flush buffered items, how large a buffer to use, etc.
Comment From: aclements
This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — aclements for the proposal review group
Comment From: jba
To be clear, the proposal is in this comment and the implementation is essentially this one.