Software development metrics

Programming languages with a historical perspective

There are many thousands of programming languages from the most widely used Java or C to the arcane or forgotten ones, like Euphoria or Ada.  If I want to write their history, I'd have to exclude most of them and select only a few.  But which ones to select, what should be the organizing principle?

I could pick the most popular languages of all time.  Or I could get a list of popular languages of each year and take the top few.  I would probably get a different list depending on the source of the statistics.  A language can be popular in the academic world with many articles and publications dealing with it and without a real presence in applications -- that was the case with Haskell until a few years ago.  Some languages are widely used in a business setting, but they won't show up in a listing of open source projects.

And why select only popular languages.  They probably have a similar story, because they are winners in a similar field.  Java and C# have a lot in common, at least technically speaking.  It may be interesting though to see if they have a different history.  But there are little-known languages which introduced some feature, but somehow didn't make it to become mainstream, like Factor, a modern, stack-based language.

It naturally gives the idea to get a list of programming paradigms and select some representatives for them.  So we could have some object-oriented and some functional languages, sprinkled with some stack-based and vector-based ones.  The list could be balanced by the type systems, so for example object-orientations would be represented by statically typed Java or C# and dynamically typed Python or Ruby.

My aim is to reflect the diversity of languages and the ideas behind them.  But I also want to show what became popular.

My wishlist (as opposed to my todo-list)

• Finish Clean code
• Learn Haskell (for great good)
• Seven languages in seven weeks
• Think and write about the history of programming languages
• Implement my array programming language in IO (or maybe just continue it in Clojure)
• Explore Prezi, learn Prezi, create prezis
• Play a little with Zsh, Git
• Play with SublimeText and decide if I want to move to it. Or stay with Vim. Or stay with Emacs+Evil. Or what?
• Make my new Mac homey 

Crash course on Mac for Linux users

Issue tracking with leiningen and midje

issue trackers are for sissies, real men use the source code

Issue tracking is such a nice concept that it deserves its own software tools. That’s how a lot of developers must have thought who created Mantis, Trac, and all the others. But something didn’t feel quite right with this approach. Issues and bugs started to live their lives independently from the code, although they were supposed to be only an aid to developers, testers, and customers.
One drawback of separating issues from code is that you don’t exactly know the status of an issue. Well, you think you know, because it’s set to “resolved” in your issue tracker. But wait, it says, it’s fixed in revision bxabxa123. It takes some mental work to figure out in which branch it’s fixed and whether that fix has been merged to your working copy. Some attempts were made to extend the DVCS and integrate issue tracking into it (dietz, bugseverywhere). Fossil and Veracity are both a DVCS that has integrated issue tracking right upfront.
An issue tracker integrated into the DVCS is a big step forward, but it’s far from perfect. The developer still has to make a context switch between bug tracking and debugging. This is even more pregnantly examplified by the fact that code snippets are copied into issue description, or the description is reformulated into code as a test case.
My bold statement is: an issue is but a test case that fails. There are some slight differences, though, but I am going to show how to handle them in a minute. Issues are poorly written from a programmer’s perspective, they are in plain English rather than in a programming language. We are also spoiled by sophisticated issue trackers, so we want a workflow, we want to assign the issue (or test case) to someone, we want to set its priority, urgency, and whatever level.
I will use clojure and midje to show how to implement issue tracking as test cases. Let’s see first how to deal with plain English description. Suppose we developed a booking system for a hotel and we forgot about superstitious guests. So our fellow tester would write a ticket in the form of a midje fact

(future-fact "handle superstitious guests"
  (booking-for-floor 13) => problem-reported)

Neither booking-for-floor, nor problem-reported exist in the code yet, but the future-fact macro hides this. When we run this snippet, it will print WORK TO DO "handle superstitious guests". (If we used fact instead of future-fact, the compiler would complain about the unresolvable symbols.) It’s the assignee’s job to convert this snippet to a proper fact that uses existing functions and variables from the system. As part of this conversion, the programmer will change future-fact to fact which shows in issue-tracking parlance that he accepted the issue.
But how can we distinguish between a real test and a bug report? Real tests should run successfully and if they fail, it means something has gone wrong. On the other hand, when a test case for a bug report fails, it only means that it’s not resolved yet. A recent feature of midje comes to rescure: metadata.
We can tag a fact with any metadata, then be selective about which facts to run and which ones to ignore. Suppose we have these facts,

(fact :bug "handle superstitious guests"
  (book-floor 12) => (throws BadLuck))


(fact "underground parking lot"
  (book-floor -2) => (throws UnavailableFloor))

We can now run

lein midje :filter -bug

to check only proper facts. (Note the minus sign before bug to filter it out.)
Metadata can be more complex, it can be used to set all the bells and whistles,

(fact :bug {:assigned "bob", :priority 4} ...)

We can write custom config files to check only facts that are bugs assigned to us with a relatively high priority. Then all we need to invoke is a single line

% lein midje :config my-important-bugs

This approach has an added bonus, it can handle granularity of bugs. In an usual issue tracker you have two options. You either write a huge ticket that contains many details, for example

• Floor -1 is parking lot, not bookable
• Floor 0 is reception desk, not bookable
• Floor 1 is bookable
• Floor 2 is bookable
• … (you get the point)
• Floor 13 is not bookable
• …
• Floor 20 is the top floor
• Floor 21 is not bookable
• …

If all these go into a single ticket, it’s pretty difficult to resolve it because of the many edge cases. If they go into separate tickets, you’ll face the tedious task of checking tickets for floors from 1 to 12 one by one.
With fact-driven issue-tracking, you group your facts the way it seems most comfortable. You can even assign a sub-fact to someone else.

(fact-group :bug {:assigned "bob"} "floors"
  (fact (book-floor 0) => FALSEY)
  (fact (book-floor 1) => truthy)
  (fact (book-floor 2) => truthy)
  (fact {:assigned "alice"} (book-floor 13) => FALSEY)
  (fact (book-floor 20) => truthy)
  (fact (book-floor 21) => FALSEY))

The 20 minute, 20 line rule

Master Turbid says, every unit of work you do should consist of less than 20 lines of code and it should be done in less than 20 minutes. You may need to think first how to achieve it. This thinking may take more than 20 minutes, and taking notes of it may take more than 20 lines. Then your unit of work is thinking and that's what should be under 20 lines and 20 minutes. (You may want to change the 20 lines according to the expressiveness of your programming language.)

You probably have a gut feeling in most cases when you are about to start a piece of work that's bound to take more time/code. Like having a quick meeting with five people on some strategic issues; it'll probably take longer unless they are focused and disciplined. Writing a wiki engine from scratch will probably take more than 20 lines; unless you are pretty skilled or you have a wiki library at hand.

In some cases you will say to yourself, "It can be done in a few lines of code, well, with some luck". Or you say to yourself, "It must be easy and I don't want to spend more time with it anyway". Or you even say to yourself, "Let's just put together something quickly and then see what comes next". Resist the temptation. Most flaws in software stem from this trigger-happy approach. If you don't exactly understand what you are doing, if the concepts in your head are not clear enough, bad things are bound to happen. You will get stuck or you will not even notice that you get stuck and cover it with some hazy code. And the sad truth about most programmers is that we can't understand large concepts clearly. Those 20 lines of code is the size that fits into our head.

If you follow this rule, you'll live a happy life. Most of your 20 minute plans will come true, you will accomplish what you aim at. Your code will be built on many successful mini-projects.
Once you mastered this rule, there is still room for improvement. You can just sit and imagine the 20 lines before touching the keyboard. Imagine the exact location of the lines: into which file and which function will they go. If you have to modify existing code, what lines have to be changed and in what way? Suppose that you don't have a keyboard, all you can do is tell a non-programmer over the phone what to do.

20 lines will seem incredibly long at first. It's also incredibly difficult to imagine code changes without looking at the code. Especially if you are spoiled by over-intelligent IDEs that spare you the burden of knowing your code intimately.

Master Turbid smiles and says, there is nothing new in this idea. Related concepts were already expressed by other masters, such as Master Pomodoro and Master Arc.

Literate programming: edit a single file

My idea of proper literate programming is that I write code as I write a novel. It's a single file that I write mostly in a linear way. When I make jumps, I warn the reader that I'll make a jump. So the whole code can be read from cover to cover, so to speak. Reading the code follows my way of thinking, how I changed my mind, how I added code to implement new requirements, how I fixed bugs, etc. The following snippet shows two example sessions of such a code editing. There is no tool yet to produce proper, working source code from it.

; I just want a simple hello-world function.  (Note the ! in next
; line: it shows it is an editing command, not part of the final code)
;! (open "src/hello/core.clj")
; The file is opened.  If the file or its parent directories don't
; exist, they are created.
(ns hello.core)

(defn greet
  "Just return a greeting"
  []
  "Hello")

; It's time now to test what we've just written
;! (open "test/hello/core_test.clj")
(ns hello.core-test
  (:use
   [clojure.test :only [deftest is]])
  (:require
   [hello.core :as core]))

(deftest greeting-test
  (is
   (= "Hello" (core/greet))))

; That's it.  We are done, we can commit our code...
;----------------------------------------------------
; The customer wants our greeting to be able to accept a `fellow`
; argument to whom the greeting is addressed.  So we have to add a
; test case for that.  Since code is data, we just have to refactor
; it.  It's usually done in an editor without much thought beforehand.
; You may use some support from the editor, like `paredit`.  But I
; want to do differently this time.  I'm coding yet, just thinking
; about what I would do in the editor.  This thinking happens to be
; some code too.

; Open the file with the greeting test (it already exists now).
;! (open "test/hello/core_test.clj")
; Find the test.
;! (find 'deftest 'greeting-test)
; Go to the first assertion
;! (zoom 'is)
; We want to append a new test case at the same level
;! (up)
;! (append-next-block)
(is
 (= "Hello, Mary" (core/greet "Mary")))
; You can guess what the final code looks now.

; We can do something more difficult now, change the greeting function
; itself.  We want to have both a 0, and a 1 argument version.
; Now you know how to navigate where we want to change the code.
;! (open "src/hello/core.clj")
;! (find 'defn 'greet)
;! (zoom '[])
; We want to make this: [] "Hello" => ([] "Hello")
;! (wrap-round)
; It just wraps the arg vector.  We have extend the paren to
; include the "Hello" part.
;! (slurp-forward)
; We are done with the 0 argument version.  We just have to add
; code for 1 one argument.
;! (append-next-block)
([fellow]
   (str "Hello, " fellow))

; This modification is done, we can commit the changes.

And this is how the files would look now:

(ns hello.core)

(defn greet
  "Just return a greeting"
  ([]
    "Hello")
  ([fellow]
     (str "Hello, " fellow)))

(ns hello.core-test)

(deftest greeting-test
  (is
   (= "Hello" (core/greet)))
  (is
   (= "Hello, Mary" (core/greet "Mary"))))

Organized? Take a break!

I tried all the trendy ways to get organized, be productive, and manage my time. I followed Stephen Covey's seven habits. I was Getting Things Done according to David Allen. I lived by Mark Forster's autofocus. I even loved some of these approaches. But I noticed that the novelty of any method eases off after a while. I have to keep myself on the track, or get back to it if I lost it. Some methods are stricter than others, but they are all methods with rules you are to follow to achieve what you have in your mind. Autofocus for instance is quite relaxed in this respect: you perform any task as long as you feel like doing it; but that's a rule anyway.

I've been doing Pomodoro for a few months now. Today was the first day when I heard a tiny voice in my head. No, I don't want to draw the little boxes, I don't want to plan my day, leave me alone. So I'm going to skip it today, just do my job without further productivity rituals. And it feels right. Here is my advice, whatever method you're using, take a break every now and then, maybe once a week, and forget about productivity even at your work.

One large file or multiple small files

There is an age-old debate that goes back before the age of computers: shall we keep stuff in one large file or in multiple small files? Do you prefer a travel jacket with plenty of pockets or you rather take huge bag with you?


When you want to decide between N x 1 and 1 x N, there are more dimensions to take into account.  There are some practical considerations.  Many programming languages want to treat abstract modules and concrete files interchangeably, having a one-to-one mapping between a module and a physical file.  Revision control systems also work on a per-file basis, so it's more meaningful to have many diffs for more small files, because it gives a more specific location of the change than having a list of changes with large line numbers for a single file.

But we tend to have two modes during development.  We want to get the big picture and navigate to a certain piece of code quickly.  Other times we want to zoom into a method body and hide everything else that clutters the view.  Code folding is one way to switch between these two modalities, at least at the file level.  Modern IDEs try and add more tools to handle this duality, such as a tree view of all modules or a view of variables and functions within a module.  Clicking on a module or a function opens it in another pane, the editor window.

I think I have an answer now, it least for files. It's not a working implementation, just a description of a program I hope someone would implement pretty soon.

The idea is that you keep everything in small files and define views on them. Views are bi-directional. A change in a file is reflected in the view. You can also edit the view and the modifications are propagated back to the original file(s). To make this work, we'll need three types of files,

• The small files which may be any kind of text files
• The view file which is a large text file containing relevant snippets from the small files and a little meta-data about the origin of each snippet
• And a template file which specifies which small files should be processed for a view and what snippets should be included

A template file in its simplest form looks like this

Something to think about
*** find:idea/*.md grep:^# .*
+ add me

It has special lines.

The view file looks like this

Index: languages/ini.js
===================================================================
--- languages/ini.js    (revision 199)
+++ languages/ini.js    (revision 200)
@@ -1,8 +1,7 @@
 hljs.LANGUAGES.ini =
 {
   case_insensitive: true,
-  defaultMode:
-  {
+  defaultMode: {
     contains: ['comment', 'title', 'setting'],
     illegal: '[^\\s]'
   },

*** /path/to/original timestamp
--- /path/to/new      timestamp
***************
*** 1,3 ****
--- 1,9 ----
+ This is an important
+ notice! It should
+ therefore be located at
+ the beginning of this
+ document!

! compress the size of the
! changes.

 It is important to spell

yeah...