The Clojure debug-repl - Dec 7, 2009

Intro

Every time I stick a println into some Clojure code to debug it, I think to myself, "This is Lisp! I should be able to insert a repl here!"

The problem is of course that Clojure's eval function doesn't know about the surrounding lexical scope. So I started asking myself, what is the simplest change I could make to Clojure to support an eval that understands that scope? Then I tried to implement it.

Basically, here's what I came up with.

1. Modify the Clojure compiler so that when a flag is turned on, it stores references to the lexical scope in a dynamic var. Thus, each time the compiler creates a new lexical scope, it also emits the byte code to push a hash-map with the details onto the var. When that scope ends, the byte code for popping the hash-map off the var is emitted.

2. Then in Clojure proper, add a special version of eval that uses that var. It wraps the form being eval'ed in a "let" that emulates the original lexical scope, something like this:

        `(eval
            (let [~@(make-let-bindings
                     (:lexical-frames (var-get (resolve context))))]
              ~form))

With those two pieces, it's straight-forward creating a "debug-repl" that understands the surrounding lexical scope.

Use

The interface is pretty simple:

"(use 'clojure.debug)" loads it, and "(debug-repl)" invokes it. You'll need to turn on lexical frame capture by surrounding your code with "with-lexical-frame". You can also use the convenience macros, "defn-debug", "defmacro-debug", and "deftest-debug" to wrap the corresponding def form in "with-lexical-frame".

That's about it. When you enter the debug-repl, the regular repl prompt will be replaced with

dr =>

Some examples will make it clearer:

user=> (use 'clojure.debug)
user=> (with-lexical-frames
          (let [c 1 d 2]
            (defn a [b c]
              (debug-repl)
                   d)))
#'user/a
user=> (a 22 (java.io.File. "/"))
dr => b
22
dr => c
#<File />
dr => d
2
dr => (* b d)
44
dr => (seq (.listFiles c))
(#<File /lost+found> #<File /var> #<File /media> #<File /etc> #<File /cdrom> ...
dr => 2
user=>

You can also use the "get-context" macro to save the context to a global var and debug it separately after running the code that created it:

user=> (use 'clojure.debug)
user=>  (with-lexical-frames
          (let [c 1 d 2]
            (defn a [b c]
              (get-context saved-context)
              d)))
#'user/a
user=> (a 22 (java.io.File. "/"))
2
user=> (debug-repl saved-context)
dr => b
22
dr => c
#<File />
dr => d
2
dr => nil
user=>

For a more interesting example, try adding the debug-repl to the contrib repl-utils "member-details" function, (which is used by the "show" function) like so:

diff --git a/src/clojure/contrib/repl_utils.clj b/src/clojure/contrib/repl_utils.clj
index 2864179..fc64e04 100644
--- a/src/clojure/contrib/repl_utils.clj
+++ b/src/clojure/contrib/repl_utils.clj
@@ -41,3 +41,4 @@

-(defn- member-details [m]
+(use 'clojure.debug)
+(defn-debug member-details [m]
   (let [static? (Modifier/isStatic (.getModifiers m))
@@ -53,2 +54,3 @@
                    (str (.getSimpleName (.getType m))))))]
+    (debug-repl)
     (assoc (bean m)

Then explore it by invoking show like this:

user=> (use 'clojure.debug)
user=> (ns clojure.contrib.repl-utils)
clojure.contrib.repl-utils=> (use ' clojure.contrib.repl-utils)
clojure.contrib.repl-utils=> (show Object)
dr => ctor?
false
dr => m
#<Method public final native void java.lang.Object.wait(long) throws java.lang.InterruptedException>
dr =>  (bean m)
{:genericReturnType void, :declaringClass java.lang.Object, :typeParameters ...
dr =>  [(not static?) method? (sortable text)]
[true true "wait : void (long)0000"]
dr =>     (assoc (bean m)
           :sort-val [(not static?) method? (sortable text)]
           :text text
           :member m)
{:sort-val [true true "wait : void (long)0000"], :genericReturnType void, :text ...
dr =>

Getting/Building the debug-repl source

The source is in the debug-repl branch of my Clojure fork on github. Get it like so:

git clone git://github.com/GeorgeJahad/clojure.git
cd clojure
git checkout --track -b debug-repl origin/debug-repl
ant

Limitations

Doesn't work with Slime repl yet

The debug-repl doesn't currently integrate properly with the slime-repl, (I think because of how Slime manages IO redirection,) so you'll have to invoke it from a regular repl. This is high on my list of things to fix. Hope to have a solution soon.

Let frames aren't generated till the end of the binding vector

As a simplication, I chose for now not to post the lexical frame for a let binding until after the entire binding is complete. This means putting a debug-repl somewhere within a let binding vector is no different from putting it immediately before the start of that let binding. I want to fix this fairly soon.

Transients

I haven't made any attempt to handle as they are changed.

Letfn bindings aren't yet stored.

Just haven't gotten around to it yet, (since I don't use letfn).

Comments/Suggestions

Send any comments/suggestions to George Jahad at "george-clojure at blackbirdsystems.net" or to the main clojure mailing list: http://groups.google.com/group/clojure