Literate programming: edit a single file

My idea of proper literate programming is that I write code as I write a novel. It's a single file that I write mostly in a linear way. When I make jumps, I warn the reader that I'll make a jump. So the whole code can be read from cover to cover, so to speak. Reading the code follows my way of thinking, how I changed my mind, how I added code to implement new requirements, how I fixed bugs, etc. The following snippet shows two example sessions of such a code editing. There is no tool yet to produce proper, working source code from it.

; I just want a simple hello-world function.  (Note the ! in next
; line: it shows it is an editing command, not part of the final code)
;! (open "src/hello/core.clj")
; The file is opened.  If the file or its parent directories don't
; exist, they are created.
(ns hello.core)

(defn greet
  "Just return a greeting"
  []
  "Hello")

; It's time now to test what we've just written
;! (open "test/hello/core_test.clj")
(ns hello.core-test
  (:use
   [clojure.test :only [deftest is]])
  (:require
   [hello.core :as core]))

(deftest greeting-test
  (is
   (= "Hello" (core/greet))))

; That's it.  We are done, we can commit our code...
;----------------------------------------------------
; The customer wants our greeting to be able to accept a `fellow`
; argument to whom the greeting is addressed.  So we have to add a
; test case for that.  Since code is data, we just have to refactor
; it.  It's usually done in an editor without much thought beforehand.
; You may use some support from the editor, like `paredit`.  But I
; want to do differently this time.  I'm coding yet, just thinking
; about what I would do in the editor.  This thinking happens to be
; some code too.

; Open the file with the greeting test (it already exists now).
;! (open "test/hello/core_test.clj")
; Find the test.
;! (find 'deftest 'greeting-test)
; Go to the first assertion
;! (zoom 'is)
; We want to append a new test case at the same level
;! (up)
;! (append-next-block)
(is
 (= "Hello, Mary" (core/greet "Mary")))
; You can guess what the final code looks now.

; We can do something more difficult now, change the greeting function
; itself.  We want to have both a 0, and a 1 argument version.
; Now you know how to navigate where we want to change the code.
;! (open "src/hello/core.clj")
;! (find 'defn 'greet)
;! (zoom '[])
; We want to make this: [] "Hello" => ([] "Hello")
;! (wrap-round)
; It just wraps the arg vector.  We have extend the paren to
; include the "Hello" part.
;! (slurp-forward)
; We are done with the 0 argument version.  We just have to add
; code for 1 one argument.
;! (append-next-block)
([fellow]
   (str "Hello, " fellow))

; This modification is done, we can commit the changes.

And this is how the files would look now:

(ns hello.core)

(defn greet
  "Just return a greeting"
  ([]
    "Hello")
  ([fellow]
     (str "Hello, " fellow)))

(ns hello.core-test)

(deftest greeting-test
  (is
   (= "Hello" (core/greet)))
  (is
   (= "Hello, Mary" (core/greet "Mary"))))

Organized? Take a break!

I tried all the trendy ways to get organized, be productive, and manage my time. I followed Stephen Covey's seven habits. I was Getting Things Done according to David Allen. I lived by Mark Forster's autofocus. I even loved some of these approaches. But I noticed that the novelty of any method eases off after a while. I have to keep myself on the track, or get back to it if I lost it. Some methods are stricter than others, but they are all methods with rules you are to follow to achieve what you have in your mind. Autofocus for instance is quite relaxed in this respect: you perform any task as long as you feel like doing it; but that's a rule anyway.

I've been doing Pomodoro for a few months now. Today was the first day when I heard a tiny voice in my head. No, I don't want to draw the little boxes, I don't want to plan my day, leave me alone. So I'm going to skip it today, just do my job without further productivity rituals. And it feels right. Here is my advice, whatever method you're using, take a break every now and then, maybe once a week, and forget about productivity even at your work.

One large file or multiple small files

There is an age-old debate that goes back before the age of computers: shall we keep stuff in one large file or in multiple small files? Do you prefer a travel jacket with plenty of pockets or you rather take huge bag with you?


When you want to decide between N x 1 and 1 x N, there are more dimensions to take into account.  There are some practical considerations.  Many programming languages want to treat abstract modules and concrete files interchangeably, having a one-to-one mapping between a module and a physical file.  Revision control systems also work on a per-file basis, so it's more meaningful to have many diffs for more small files, because it gives a more specific location of the change than having a list of changes with large line numbers for a single file.

But we tend to have two modes during development.  We want to get the big picture and navigate to a certain piece of code quickly.  Other times we want to zoom into a method body and hide everything else that clutters the view.  Code folding is one way to switch between these two modalities, at least at the file level.  Modern IDEs try and add more tools to handle this duality, such as a tree view of all modules or a view of variables and functions within a module.  Clicking on a module or a function opens it in another pane, the editor window.

I think I have an answer now, it least for files. It's not a working implementation, just a description of a program I hope someone would implement pretty soon.

The idea is that you keep everything in small files and define views on them. Views are bi-directional. A change in a file is reflected in the view. You can also edit the view and the modifications are propagated back to the original file(s). To make this work, we'll need three types of files,

• The small files which may be any kind of text files
• The view file which is a large text file containing relevant snippets from the small files and a little meta-data about the origin of each snippet
• And a template file which specifies which small files should be processed for a view and what snippets should be included

A template file in its simplest form looks like this

Something to think about
*** find:idea/*.md grep:^# .*
+ add me

It has special lines.

The view file looks like this

Index: languages/ini.js
===================================================================
--- languages/ini.js    (revision 199)
+++ languages/ini.js    (revision 200)
@@ -1,8 +1,7 @@
 hljs.LANGUAGES.ini =
 {
   case_insensitive: true,
-  defaultMode:
-  {
+  defaultMode: {
     contains: ['comment', 'title', 'setting'],
     illegal: '[^\\s]'
   },

*** /path/to/original timestamp
--- /path/to/new      timestamp
***************
*** 1,3 ****
--- 1,9 ----
+ This is an important
+ notice! It should
+ therefore be located at
+ the beginning of this
+ document!

! compress the size of the
! changes.

 It is important to spell

yeah...