William Huster, Software Architect

Practice Command Line Programming with wtf.py

Source Code on Github

Most programmers are familiar with this iconic comic:

WTF Source

And anyone who has ever done code review or even just tried to read code knows how true this is.

Well, I’ve been doing more code reviews lately, and I found myself wishing I had an easy way to record all those moments that just make me go: “WTF”.

Seriously, though, it’s useful to keep a log of your thoughts while doing code review for later summary, and I thought it could be even better to timestamp those thoughts. That way, I could quantify exactly how much of my life had been wasted on crappy code. Inspired by the comic above, I also wanted to generate a WTF/min score as a useful (stretching that word) code review benchmark.

It’s also just plain cathartic to pound out a WTF or two if ever you find yourself looking at very. bad. code.

The Basics

The core of wtf.py is very simple: when executed, the script appends a new line to a text file containing an ISO-formatted timestamp and optional message. Read the code comments to understand what’s going on.

Let’s start by writing a minimum-viable version of wtf.py:

#!/usr/bin/env python
# ^ Make sure this is the first line of the file. This instructs your
# shell to use Python to execute this file. In Linux, first make the file
# executable with the command:
#
#    chmod +x wtf.py
#
# and then you will be able to run the script like so:
#
#    ./wtf.py
#
# Of course, you can also always run the script with python directly:
#
#    python wtf.py
#

"""wtf.py Minimum Viable"""

# We'll need the datetime module to create timestamps
from datetime import datetime


# The `write_wtf` function:
# Putting logic inside a function is always a good idea. This will
# allow other developers to import the module and use this function.
def write_wtf():
    # By default, wtf.py writes to a file (let's call it a "WTF Log File")
    # called 'wtfs.txt' in the current directory.
    outfile = 'wtfs.txt'

    # Next, we open the output file in append-only mode using 'a' as
    # the second parameter. The '+' sign tells Python to create a new file,
    # if it doesn't already exist.
    with open(outfile, 'a+') as f:
        # Get the current timestamp
        now = datetime.now()

        # Write the timestamp to the file in ISO Format and create
        # a line break with '\r\n'.
        f.write(now.isoformat())
        f.write('\r\n')


# `if __name__ == "__main__"` code block:
# Code that should only be run from the command line should go inside
# this block. That way, it will not be unintentionally executed if
# the module is imported by another Python program.
if __name__ == "__main__":
    write_wtf()

And that’s all we need to get started recording WTF moments. Now just run python wtf.py. It immediately executes the write_wtf() function. When you look at wtfs.txt, you will be able to see the exact moments that caused you to flip your lid — with microsecond precision.

Quick note on the timestamps: The ISO timestamps generated by this script will be in local time and ‘naive’, meaning they will not record any timezone. This is in the interest of simplicity and to avoid introducing code dependencies like python dateutil, which is better at parsing datetimes with timezones. When parsing WTF Log Files, wtf.py will assume that timestamps in the same log file have the same timezone… This kind of assumption is usually not safe, but, hey. We’re just having fun here.

Adding Messages

That’s all well and good, but suppose you also want to record the cause of your fury?

This is a good opportunity to use Python’s built-in argparse library to practice defining argument options and parsing command-line arguments. That may seem intimidating, but it’s actually quite easy to use!

# ... Code ommitted

# We'll need argparse
import argparse

# Here we set up the argument parser class and define the optional
# '--message' option. In typical command line style, the description
# and help text can be displayed with the '-h' option:
#
#     python wtf.py -h
#
parser = argparse.ArgumentParser(description="Easily record WTF moments.")
parser.add_argument(
    # Options prefixed with a double-dash will not be required.
    # You can also provide a short version of the option. Here
    # I've added '-m', short for '--message'.
    '-m', '--message',
    # Require that the value of message is a string
    type=str,
    help='Record the thing that made you go WTF'
)


# Notice the new 'message' parameter of the write_wtf function.
# Since it's optional, the default value will be None.
def write_wtf(message=None):
    outfile = 'wtfs.txt'

    with open(outfile, 'a+') as f:
        now = datetime.now()
        f.write(now.isoformat())

        # And here's the only other change -- if `message` has a value,
        # then add a space and the message to the current line.
        if message:
            f.write(' ' + message)
        f.write('\r\n')


if __name__ == "__main__":
    # Now we'll parse the command line arguments before writing
    # down the WTF moment. `args.message` will have a value whenever
    # the user provides the --message or -m option on the command line.
    args = parser.parse_args()
    message = args.message

    write_wtf(message)

Use your message option like so:

python wtf.py -m "Use SPACES not TABS!!!"

So now you can record both the exact moment you WTF’d and why. Amazing.

Parsing WTF Logs

Once we’ve recorded a few WTFs, we can use cat wtfs.txt or open our WTF Log File in a text editor at any time to review the contents. For example, I’d do this once I’ve finished my code review and need to summarize all of the unspeakable horrors I uncovered.

But the REAL beauty of the structured text file generated by wtf.py is that we can PARSE it and run some numbers! This is how we get to that holy grail benchmark number from the famous comic: WTFs per minute.

A typical WTF Log File will look like this

2018-05-05T13:55:02.328431
2018-05-05T13:56:08.034393 WHYYyyyy
2018-05-05T14:02:14.222576 I can't even...

etc… So the first ‘word’ of each line will be the ISO timestamp written down by the write_wtf() function. To calculate WTFs/minute, we just need to parse those timestamps into Python datetime objects, subtract the last timestamp from the first to get the total duration in minutes and then divide the total number of WTFs by the total minutes. Easy peasy. Let’s code it up.


# ... Code skipped

# Add an argument to the argument parser for reading WTF log files.
# Its value should be the name of the file to read.
parser.add_argument(
    '-r', '--read',
    help='Read a WTF log file. Outputs stats and WTFs per minute.'
)


def _parse_timestamp(timestamp):
    """Get a timezone-naive datetime object from an ISO timestamp string.

    This is a helper function for consistently parsing the timestamps in
    WTF log files. The undescore in front of the function is a common
    programming convention that indicates that this function is 'private'
    and should only be used by this module.

    In fact, if you import this module into another module using

        from wtf import *

    This function will be ignored by the import. Defining functions like
    this is in Python is nothing more than a matter of style preference,
    since no code is truly private or protected. You can think of it as
    a message to other developers, saying, "You probably shouldn't use
    this function, unless you really know what you're doing."

    Args:
        timestamp (str): Timezone-naive ISO-formatted datetime string.
    """
    ISO_FMT = '%Y-%m-%dT%H:%M:%S.%f'
    return datetime.strptime(timestamp, ISO_FMT)


def get_stats(filename):
    """Read a WTFs log file and compute statistics, including:
         - Total WTFs
         - Total Duration
         - WTFs/minute

    Args:
        filename (str): Path to the WTF log file.

    Returns:
        dict: Dictionary of statistics, including:
    """
    conent = []
    # Open the file for reading and get all of the lines as a list
    with open(filename, 'r') as f:
        content = f.readlines()

    # Strip the line endings and whitespace from each line
    content = [x.strip() for x in content]

    # Grab just the timestamps from each line. After splitting on
    # spaces, this should always be the first element in the list.
    # ... If this were a serious utility, we might add some error
    # handling in here to detect malformed WTF log files and show
    # a friendly error message.
    timestamps = [line.split(' ')[0] for line in content]

    # Calculate the duration from the first WTF moment to the last.
    start = _parse_timestamp(min(*timestamps))
    end = _parse_timestamp(max(*timestamps))
    time_diff = end - start
    total_minutes = time_diff.seconds / 60

    # Prevent division by zero for timestamps that are very close.
    # (this is a bit of a hack)
    if total_minutes == 0:
        total_minutes = 1

    # Return a dictionary of key metrics!
    return {
        'total': len(timestamps),
        'duration': total_minutes,
        'wtfs_per_min': len(timestamps) / total_minutes,
    }


# ... Code skipped

# Now we're also getting the value for args.read. If that option is used,
# then the script invokes the get_stats() function on the provided filename.
# Otherwise, it writes a new WTF moment, just like before.
if __name__ == "__main__":
    args = parser.parse_args()
    message = args.message
    readfile = args.read

    if readfile:
        # The --read option should not be combined with the
        # --message option. If the user tries this, raise a
        # ValueError and show a nice message.
        if message:
            raise ValueError(
                'The --read option cannot be used with '
                'the --message option.'
            )
        stats = get_stats(readfile)
        # Print out those stats!
        print('Total WTFs:', stats['total'])
        print('Duration (minutes):', str(round(stats['duration'], 3)))
        print('WTFs/min:', str(round(stats['wtfs_per_min'], 3)))
    else:
        write_wtf(message, outfile)

Beautiful. Now let’s use our new option to see how the latest code review went:

$ python wtf.py -r wtfs.txt
Total WTFs: 35
Duration (minutes): 20.54
WTFs/min: 1.704

More than one WTF per minute? Oh, boy… That’s a pretty bad score. I might have never known without this script. Thanks, wtf.py!

Next Steps

Yes, this is a pretty exhaustive explanation of a fairly simple tool, but I hope it helps a few beginners understand the power of the command line and a simple text file. This is easy to forget in a world of web apps, frameworks, database servers, containerization, etc, etc. By staying focused on just the basics, you can whip up something useful to you in a matter of minutes.

And what more could be done with this? Here are a couple ideas:

  • Add an option for specifying the name of the output file (instead of just wtfs.txt)
  • Integrate this with a text editor like sublime text to record comments against actual lines of code.
  • Turn this into a social web or mobile app. What if people could share their WTF moments throughout the day with some richer media: geolocation, images, video, etc.?