Logging Like a Pro in Go

codeHelm on youtube

In the last article, we talked about interpreting profiling output and what developers can do to avoid bottlenecks in their projects. Today, we are exploring logging. Go leverages the runtime and time packages to generate a time-stamped, color-coded checkpoint string that includes the method name, file name, and line number as shown below.

So what’s going on here?

Go’s runtime system has these cool functions called `runtime.Caller` and `runtime.FuncForPC` that let you peek into what the program’s doing at any moment. They’re super handy for figuring out the call stack and getting info about the functions involved.

You would define a function that utilizes the runtime package’s `Caller` function to retrieve essential debugging information.

• TImestamp 2024–12–21 11:34:23.673875577 +0300 EAT
• Function name main.method1
• FIle name logging1.go
• Line number 19

This includes the program counter (PC), the file name, and the line number from which the function was called. Additionally, the function name can be extracted using the `runtime.FuncForPC` function.

func checkPoint() string { 
  pc, file, line, _ := runtime.Caller(1) 
  return fmt.Sprintf("\033[31m%v %s %s %d\x1b[0m", time.Now(), runtime.FuncForPC(pc).Name(), path.Base(file), line) 
}

The method where you call the function above is the one logged (method1 in this instance)

func method1() {  
  log.Println(checkPoint()) 
}

This output is formatted into a string that includes a timestamp (from time.Now), function name, file name (which is extracted using path.Base), and line number. ANSI escape codes are used to add red color to the output.

But…

This is for basic usage. What if you want to add support for different colors based on severity levels (like info, warning, error), you would do something like:

func checkPoint(level string) string {
    color := "\033[32m" // Default to green for info
    if level == "ERROR" {
        color = "\033[31m" // Red for errors
    } else if level == "WARN" {
        color = "\033[33m" // Yellow for warnings
    }
    pc, file, line, _ := runtime.Caller(1)
    return fmt.Sprintf("%s%v %s %s %d\x1b[0m", color, time.Now(), runtime.FuncForPC(pc).Name(), path.Base(file), line)
}

This function will dynamically apply colors based on severity levels.

Integration with External Logging Frameworks

For projects that need structured or persistent logs, you can achieve better flexibility and performance by integrating with a logging framework like Logrus or Zap. You can then send logs to services like Elasticsearch, Splunk, or Datadog for centralized monitoring.

Strap in as we are going to get our hands dirty with Logrus and Logstash Hook to try and send logs to Elasticsearch. 😎

I gotta say setting up Elasticsearch locally on a Linux OS was fun! 😀

If you want to follow along how I did it, check out this article I did. It is important to note that this is only for development purposes and should never be used for production environments.

The installation of Logrus, a structured logger for Go, is the first step.

go get github.com/sirupsen/logrus

Below custom hook transforms Logrus entries into JSON format and transmits them to an Elasticsearch index, usually through HTTP APIs. This allows for high-speed search and analytics, as each log is stored as a document within the Elasticsearch index. Efficient filtering is achieved by indexing logs based on criteria such as date, environment (e.g., production, staging), or service name.

// Add custom fields (method name, file, line etc) to the logger 
func addLogFields(logger *logrus.Entry) *logrus.Entry { 
  pc, file, line, _ := runtime.Caller(2) 
  function := runtime.FuncForPC(pc).Name() 
  return logger.WithFields(logrus.Fields  {
    "timestamp": time.Now(), 
    "function": function, 
    "file": path.Base(file), 
    "line": line, 
})}

So basically, by default, Elasticsearch 8.x versions enforce security, including HTTPS communication, even in single-node setups. You will see something like:

"log.level": "WARN", "message":"received plaintext http traffic on an https channel, closing connection”

You will need to either configure your project to use HTTPS or disable Elasticsearch’s HTTPS requirement during setup like so:-

client, error := elastic.NewClient(
    elastic.SetURL("https://localhost:9200"),
    elastic.SetBasicAuth("elastic", "<password>"),
    elastic.SetSniff(false),
    elastic.SetHealthcheck(true),
)
if error != nil {
    log.Fatalf("Error creating Elasticsearch client: %s", error)
}

The default self-signed certificate in Elasticsearch may lead to certificate verification problems. Disabling certificate verification within your application is an option.

elastic.SetHttpClient(&http.Client{
    Transport: &http.Transport{
        TLSClientConfig: &tls.Config{
            InsecureSkipVerify: true,
        },
    },
})

When you run:

go run logging_elastic.go

You should see this nicely color-coded Logrus output 👌

Check out the full code here

When you run:

curl -k [PASSWORD] https://localhost:9200

You should see this JSON response:

Open your browser and go to:

http://localhost:5601

You should see the Kibana dashboard. If Kibana is secured, log in using the username and password (e.g., elastic and the password you set up earlier).

Again, most of the steps here are in setting up elasticsearch and kibana which I have documented here for your reference with the hurdles and errors I received.

Below is how the data is visualized from documents and field statistics tabs on the dashboard

From my examples above, the system monitors several activities, including:

• Application lifecycle events (e.g., starting or ending the application)
• User actions (e.g., ‘User logged in by john_doe’)
• Warnings about potential operational issues (e.g., ‘Large file upload detected’)
• Critical errors (e.g., ‘disk full’)

Developers can leverage this information for various purposes, such as:

• Debugging application issues (e.g., resolving ‘disk full’ errors)
• Monitoring application performance and usage (e.g., tracking user logins and file uploads)
• Generating reports and metrics

Thank you 🙏 and see you soon!