Radu Cotescu's professional blog

g33k w17h pa45510n

Coloured Log Outputs on Unix/Linux

There are various times when a software engineer needs to closely monitor log output in real-time. More often than not the engineer is actually interested in the output of multiple logs at the same time. Given the amount of information that an application can output to log files at any one time, filtering the important information from bogus entries becomes frustrating and tedious.

One of the solutions I’ve come across recently is to colorize the log output according to the log entries’ severity.

GUI half-baked options

On GNOME-based Linux distributions there’s GNOME System Log Viewer, a GUI application that can be used to monitor logs in real-time. One nice functionality that it provides are filters which can be configured to match an expression or a regular expression; the filters allow colorizing the output according to what they match. +1 for Linux.

On Mac OS X Console.app, although is able to filter information faster and easier than GNOME System Log Viewer, is not able to highlight the lines of interest using colours. Which is a shame, given the fact that more and more developers are migrating to Macs.

tail and the power of Bash

Every Unix/Linux user has used tail at least once (though I have my doubts regarding some of the Mac users :P). The advantage of tail is that it allows piping its output to whatever program able to consume information from stdin.

Enter awk

awk is an interpreted programming language perfect for extracting data from text files. Given the fact that awk can read from stdin, it’s quite trivial to pipe data to it from which to extract more meaningful information. Combine this with a little Bash scripting (needs Bash 4 due to associative arrays) and we have a nice and easy way to analyze logs in real-time using colour-coded output.

ctail with awkClone on GitHub
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#!/usr/local/bin/bash
TAIL=`which tail`
AWK=`which awk`
if [[ -z $TAIL ]]; then
    echo -e "Cannot find tail executable.\n"
    exit 1
fi
if [[ -z $AWK ]]; then
    echo -e "Cannot find awk executable.\n"
    exit 1
fi

CWD=`dirname $0`

declare -A log
log["INFO"]="BWhite"
log["ERROR"]="BRed"
log["WARN"]="BYellow"
log["DEBUG"]="BGreen"

if [[ -f "$CWD/ctail.config" ]]; then
    source "$CWD/ctail.config"
fi

declare -A colours
# Regular
colours["Black"]="\033[0;30m"
colours["Red"]="\033[0;31m"
colours["Green"]="\033[0;32m"
colours["Yellow"]="\033[0;33m"
colours["Blue"]="\033[0;34m"
colours["Purple"]="\033[0;35m"
colours["Cyan"]="\033[0;36m"
colours["White"]="\033[0;37m"
#Bold
colours["BBlack"]="\033[1;30m"
colours["BRed"]="\033[1;31m"
colours["BGreen"]="\033[1;32m"
colours["BYellow"]="\033[1;33m"
colours["BBlue"]="\033[1;34m"
colours["BPurple"]="\033[1;35m"
colours["BCyan"]="\033[1;36m"
colours["BWhite"]="\033[1;37m"
# High Intensity
colours["IBlack"]="\033[0;90m"
colours["IRed"]="\033[0;91m"
colours["IGreen"]="\033[0;92m"
colours["IYellow"]="\033[0;93m"
colours["IBlue"]="\033[0;94m"
colours["IPurple"]="\033[0;95m"
colours["ICyan"]="\033[0;96m"
colours["IWhite"]="\033[0;97m"
# Bold High Intensity
colours["BIBlack"]="\033[1;90m"
colours["BIRed"]="\033[1;91m"
colours["BIGreen"]="\033[1;92m"
colours["BIYellow"]="\033[1;93m"
colours["BIBlue"]="\033[1;94m"
colours["BIPurple"]="\033[1;95m"
colours["BICyan"]="\033[1;96m"
colours["BIWhite"]="\033[1;97m"

AWK_EXPRESSION=""
for i in "${!log[@]}"; do
    AWK_EXPRESSION="/$i/ {print \"${colours[${log[$i]}]}\" \$0 \"\033[39m\"} $AWK_EXPRESSION"
done

$TAIL "$@" | $AWK "$AWK_EXPRESSION"

A ctail.config file can be created in the same folder with ctail to override the default colours and log levels associations.

The advantage of using awk is that it’s present on most of the available Linux distributions and UNIX flavours and that it’s reasonably fast.

Option no. 2: ack

Another tool used for filtering information is ack, a tool “designed as a replacement for 99% of the uses of grep”. A similar colorized tail script can be written like:

ctail with ack
1
2
3
4
5
6
7
8
#!/bin/bash

tail "$@" \
| ack --flush --passthru --color --color-match=red    "^.*\*ERROR\*.*" \
| ack --flush --passthru --color --color-match=yellow "^.*\*WARN\*.*" \
| ack --flush --passthru --color --color-match=green  "^.*\*INFO\*.*" \
| ack --flush --passthru --color --color-match=white  "^.*\*DEBUG\*.*" \
| ack --flush --passthru --color --color-match=white  "^.*\*TRACE\*.*"

Credits for this solution go to Alex Klimetschek, one of my colleagues from Adobe. ack does not come standard on most Linux distributions, nor on Mac OS X; however, installing it is usually a few commands away with your favourite package manager.

In the end

Being an avid terminal user, I find the ctail script (in any of its variations) a better tool than its half-baked GUI alternatives. Furthermore, ctail can also be used on servers with very little setup.

Do you know of any other interesting ways for highlighting log messages?

Code, Linux

« A useful rant on how people write Java code Remapping Keys in Ubuntu 14.04 »

Comments