There are many logging libraries for Go, and they all have their pros and cons. Some are more verbose, some are more flexible, some are more performant, and some are more opinionated. But none of them fit our needs. So we wrote our own extensible, colorful, and easy-to-use logger.
Background
We have been reliably using the standard library’s log package. It works great
and has a simple API. However, it doesn’t have structured logging, log levels,
and is not customizable. This means that to use structured logging, we
have to introduce breaking changes to our codebase.
Another problem we were facing was excessive logging which in turn drove up our infrastructure cost. We were logging debugging logs on our production instances. Using leveled logging, we could set different levels per environment and avoid logging on specific levels thus driving the cost down.
Readability was also important to us. We were looking for a library that prints pretty logs that are easy to read in a development environment. Along with having a simple, flexible API that accepts different types and values.
The Hunt for a Logger
There are many great logging libraries for Go. We tried switching to logrus, apex/log, and go-kit/log before coming up with our own logger.
Logrus was intuitive, easy to migrate to, and fully
API-compatible with the standard library logger. Given the widespread use of
logrus and the use of structured logging, this was our ideal
option. We can start the migration from the standard library logger by replacing
all the log imports with log "github.com/sirupsen/logrus". However, given
that the project is now in maintenance mode, meaning they won’t be introducing
new features, we continued our search for the most suitable logger.
We quickly stumbled upon apex/log. Apex/log has a similar & simpler API to logrus. It was inspired by logrus. However, it wasn’t fully API-compatible with the standard library logger. At this point, we figured, to support structured logging and to encourage ourselves to use structured logging, we had to introduce breaking changes to our codebase.
For example, to make the following log example structured, we can write it in
apex/log using the log.Fields type:
// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)
// structured & leveled: "apex/log"
log.WithFields(log.Fields{
"app": app,
"env": env,
}).WithError(err).Error("unable to process data")
This was not what we were looking for in terms of ease of use. We find introducing more types and methods extraneous and slows the process of writing a log statement. Plus, apex/log brings more dependencies given its centralized approach. That’s when we started looking for a minimalistic library with fewer dependencies.
Go-kit/log offers a minimal structured logging library, that simple. It offers one extensible interface:
type Logger interface {
Log(keyvals ...interface{}) error
}
Now, let’s write the above example with go-kit/log:
// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)
// structured: "go-kit/log"
logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
logger = log.With(logger, "ts", log.DefaultTimestamp, "app", app, "env", env)
logger.Log("msg", "unable to process data", "err", err)
Wait, I don’t see any log levels here?
Given the extensible and minimal architecture of go-kit/log, leveled loggers
are just another wrapper around the interface. You can use the level
package to add support for
leveled logging.
// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)
// structured & leveled: "go-kit/log"
logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
logger = level.NewFilter(logger, level.AllowInfo)
logger = log.With(logger, "ts", log.DefaultTimestamp, "app", app, "env", env)
level.Error(logger).Log("msg", "unable to process data", "err", err)
You can see this is starting to get complicated and requires some extra steps to get it going. Unlike the standard library logger, logrus, and apex/log, go-kit/log doesn’t offer a global logger instance to use. Its minimal API and interface require wrapping the logger instance in multiple layers to get the same out-of-box experience that logrus or apex/log offer.
Shout out to all of the people behind these awesome libraries:
- apex/log – A structured logger for Go.
- coder/slog – A minimal structured logger.
- go-kit/log – A minimalistic structured logger.
- golang/glog – Leveled execution logs for Go.
- hashicorp/go-hclog – A simple, structured, leveled logger.
- rs/zerolog – A zero-allocation JSON logger.
- sirupsen/logrus – A structured, pluggable logger for Go.
- uber-go/zap – A fast, structured, leveled logger.
The Charm Logger
We realized that what we need is a structured, leveled, and easy-to-use logger. It should come with a global package-wise singleton, offer multiple output formats such as text, JSON, and Logfmt, and provide an extensible interface. We also want it to be compatible with the standard library logger.
Just like the standard library version, the Charm logger comes with a global singleton that can be used with the package namespace. It comes with a set of defaults such as reporting timestamps, filtering out levels less than info, and formatting logs for the console.
// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)
// structured & leveled: "charmbracelet/log"
log.Error("unable to process data", "app", app, "env", env, "err" err)
Example of the output:
The Charm logger supports 3 different formats, TextFormatter (default)
suitable for console output, JSONFormatter and LogfmtFormatter for
production use. Use log.SetFormatter() or log.Options{Formatter: } option to
set the output format. TextFormatter uses Lip Gloss to style the
output. You can customize the styles by modifying the style definitions in
styles.go.
You can also create sub-loggers with log.With() and pass fields as key-value pairs.
logger := log.With("app", app, "env", env)
logger.Error("unable to process data", "err" err)
Want to send your logs to a file? No problem:
f, _ := os.OpenFile("log.txt", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
log.SetOutput(f)
log.SetFormatter(log.JSONFormatter) // Use JSON format
The logger also offers different options such as reporting caller location, changing the log level, setting a prefix, and changing the timestamp format. Refer to the logger interface to see all the available options.
That’s it! You can find Log, our official logger, on our GitHub. If you have any thoughts or comments, don’t be afraid to share them with us.