21 February 2023

The Charm Logger

By Ayman Bagabas

There are many logging libraries for Go, and they all have their pros and cons. Some are more verbose, some are more flexible, some are more performant, and some are more opinionated. But none of them fit our needs. So we wrote our own extensible, colorful, and easy-to-use logger.


We have been reliably using the standard library’s log package. It works great and has a simple API. However, it doesn’t have structured logging, log levels, and is not customizable. This means that to use structured logging, we have to introduce breaking changes to our codebase.

Another problem we were facing was excessive logging which in turn drove up our infrastructure cost. We were logging debugging logs on our production instances. Using leveled logging, we could set different levels per environment and avoid logging on specific levels thus driving the cost down.

Readability was also important to us. We were looking for a library that prints pretty logs that are easy to read in a development environment. Along with having a simple, flexible API that accepts different types and values.

The Hunt for a Logger

There are many great logging libraries for Go. We tried switching to logrus, apex/log, and go-kit/log before coming up with our own logger.

Logrus was intuitive, easy to migrate to, and fully API-compatible with the standard library logger. Given the widespread use of logrus and the use of structured logging, this was our ideal option. We can start the migration from the standard library logger by replacing all the log imports with log "github.com/sirupsen/logrus". However, given that the project is now in maintenance mode, meaning they won’t be introducing new features, we continued our search for the most suitable logger.

We quickly stumbled upon apex/log. Apex/log has a similar & simpler API to logrus. It was inspired by logrus. However, it wasn’t fully API-compatible with the standard library logger. At this point, we figured, to support structured logging and to encourage ourselves to use structured logging, we had to introduce breaking changes to our codebase.

For example, to make the following log example structured, we can write it in apex/log using the log.Fields type:

// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)

// structured & leveled: "apex/log"
	"app": app,
	"env": env,
}).WithError(err).Error("unable to process data")

This was not what we were looking for in terms of ease of use. We find introducing more types and methods extraneous and slows the process of writing a log statement. Plus, apex/log brings more dependencies given its centralized approach. That’s when we started looking for a minimalistic library with fewer dependencies.

Go-kit/log offers a minimal structured logging library, that simple. It offers one extensible interface:

type Logger interface {
	Log(keyvals ...interface{}) error

Now, let’s write the above example with go-kit/log:

// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)

// structured: "go-kit/log"
logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
logger = log.With(logger, "ts", log.DefaultTimestamp, "app", app, "env", env)
logger.Log("msg", "unable to process data", "err", err)

Wait, I don’t see any log levels here?

Given the extensible and minimal architecture of go-kit/log, leveled loggers are just another wrapper around the interface. You can use the level package to add support for leveled logging.

// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)

// structured & leveled: "go-kit/log"
logger := log.NewLogfmtLogger(log.NewSyncWriter(os.Stderr))
logger = level.NewFilter(logger, level.AllowInfo)
logger = log.With(logger, "ts", log.DefaultTimestamp, "app", app, "env", env)
level.Error(logger).Log("msg", "unable to process data", "err", err)

You can see this is starting to get complicated and requires some extra steps to get it going. Unlike the standard library logger, logrus, and apex/log, go-kit/log doesn’t offer a global logger instance to use. Its minimal API and interface require wrapping the logger instance in multiple layers to get the same out-of-box experience that logrus or apex/log offer.

Shout out to all of the people behind these awesome libraries:

The Charm Logger

This is fine.

We realized that what we need is a structured, leveled, and easy-to-use logger. It should come with a global package-wise singleton, offer multiple output formats such as text, JSON, and Logfmt, and provide an extensible interface. We also want it to be compatible with the standard library logger.

Just like the standard library version, the Charm logger comes with a global singleton that can be used with the package namespace. It comes with a set of defaults such as reporting timestamps, filtering out levels less than info, and formatting logs for the console.

// unstructured: golang "log"
log.Printf("starting app: %s, env: %s", app, env)
log.Printf("unable to process data: %s", err)

// structured & leveled: "charmbracelet/log"
log.Error("unable to process data", "app", app, "env", env, "err" err)

Example of the output: example

The Charm logger supports 3 different formats, TextFormatter (default) suitable for console output, JSONFormatter and LogfmtFormatter for production use. Use log.SetFormatter() or log.Options{Formatter: } option to set the output format. TextFormatter uses Lip Gloss to style the output. You can customize the styles by modifying the style definitions in styles.go.

You can also create sub-loggers with log.With() and pass fields as key-value pairs.

logger := log.With("app", app, "env", env)
logger.Error("unable to process data", "err" err)

Want to send your logs to a file? No problem:

f, _ := os.OpenFile("log.txt", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
log.SetFormatter(log.JSONFormatter) // Use JSON format

The logger also offers different options such as reporting caller location, changing the log level, setting a prefix, and changing the timestamp format. Refer to the logger interface to see all the available options.

That’s it! You can find Log, our official logger, on our GitHub. If you have any thoughts or comments, don’t be afraid to share them with us.


Read this post in your terminal with Glow:

glow -p https://charm.sh/blog/the-charm-logger.md Copied!

By Ayman Bagabas

21 February 2023

Ayman is a software developer and DevOps engineer at Charm. He designs & manages Charm infrastructure.

Lets chat!

Have a question about a command line thing you’re building? Got an idea for a new feature? Just wanna hang out? You’re always welcome in the Charm Discord.