Issue tracking with leiningen and midje

issue trackers are for sissies, real men use the source code

Issue tracking is such a nice concept that it deserves its own software tools. That’s how a lot of developers must have thought who created Mantis, Trac, and all the others. But something didn’t feel quite right with this approach. Issues and bugs started to live their lives independently from the code, although they were supposed to be only an aid to developers, testers, and customers.
One drawback of separating issues from code is that you don’t exactly know the status of an issue. Well, you think you know, because it’s set to “resolved” in your issue tracker. But wait, it says, it’s fixed in revision bxabxa123. It takes some mental work to figure out in which branch it’s fixed and whether that fix has been merged to your working copy. Some attempts were made to extend the DVCS and integrate issue tracking into it (dietz, bugseverywhere). Fossil and Veracity are both a DVCS that has integrated issue tracking right upfront.
An issue tracker integrated into the DVCS is a big step forward, but it’s far from perfect. The developer still has to make a context switch between bug tracking and debugging. This is even more pregnantly examplified by the fact that code snippets are copied into issue description, or the description is reformulated into code as a test case.
My bold statement is: an issue is but a test case that fails. There are some slight differences, though, but I am going to show how to handle them in a minute. Issues are poorly written from a programmer’s perspective, they are in plain English rather than in a programming language. We are also spoiled by sophisticated issue trackers, so we want a workflow, we want to assign the issue (or test case) to someone, we want to set its priority, urgency, and whatever level.
I will use clojure and midje to show how to implement issue tracking as test cases. Let’s see first how to deal with plain English description. Suppose we developed a booking system for a hotel and we forgot about superstitious guests. So our fellow tester would write a ticket in the form of a midje fact

(future-fact "handle superstitious guests"
  (booking-for-floor 13) => problem-reported)

Neither booking-for-floor, nor problem-reported exist in the code yet, but the future-fact macro hides this. When we run this snippet, it will print WORK TO DO "handle superstitious guests". (If we used fact instead of future-fact, the compiler would complain about the unresolvable symbols.) It’s the assignee’s job to convert this snippet to a proper fact that uses existing functions and variables from the system. As part of this conversion, the programmer will change future-fact to fact which shows in issue-tracking parlance that he accepted the issue.
But how can we distinguish between a real test and a bug report? Real tests should run successfully and if they fail, it means something has gone wrong. On the other hand, when a test case for a bug report fails, it only means that it’s not resolved yet. A recent feature of midje comes to rescure: metadata.
We can tag a fact with any metadata, then be selective about which facts to run and which ones to ignore. Suppose we have these facts,

(fact :bug "handle superstitious guests"
  (book-floor 12) => (throws BadLuck))


(fact "underground parking lot"
  (book-floor -2) => (throws UnavailableFloor))

We can now run

lein midje :filter -bug

to check only proper facts. (Note the minus sign before bug to filter it out.)
Metadata can be more complex, it can be used to set all the bells and whistles,

(fact :bug {:assigned "bob", :priority 4} ...)

We can write custom config files to check only facts that are bugs assigned to us with a relatively high priority. Then all we need to invoke is a single line

% lein midje :config my-important-bugs

This approach has an added bonus, it can handle granularity of bugs. In an usual issue tracker you have two options. You either write a huge ticket that contains many details, for example

• Floor -1 is parking lot, not bookable
• Floor 0 is reception desk, not bookable
• Floor 1 is bookable
• Floor 2 is bookable
• … (you get the point)
• Floor 13 is not bookable
• …
• Floor 20 is the top floor
• Floor 21 is not bookable
• …

If all these go into a single ticket, it’s pretty difficult to resolve it because of the many edge cases. If they go into separate tickets, you’ll face the tedious task of checking tickets for floors from 1 to 12 one by one.
With fact-driven issue-tracking, you group your facts the way it seems most comfortable. You can even assign a sub-fact to someone else.

(fact-group :bug {:assigned "bob"} "floors"
  (fact (book-floor 0) => FALSEY)
  (fact (book-floor 1) => truthy)
  (fact (book-floor 2) => truthy)
  (fact {:assigned "alice"} (book-floor 13) => FALSEY)
  (fact (book-floor 20) => truthy)
  (fact (book-floor 21) => FALSEY))