swank-cdt: Using Slime with the Clojure Debugging Toolkit

What is it?

Swank-cdt enables swank-clojure to use the CDT, as a debugger backend. The CDT is a Clojure command line debugger which allows you to debug through the Java Debug Interface.

With swank-cdt, you can step, set breakpoints, catch exceptions, and eval clojure expressions, in the context of the current stack frame, from within the sldb buffer.

Setup

With clojure-jack-in

No setup is required if you are using clojure-jack-in and swank-clojure 1.4.0. It should all just work.

With slime or slime-connect

Using slime/slime-connect requires more significant modification of your project.clj, e.g:

  :dev-dependencies [[swank-clojure "1.4.0"]]
  :jvm-opts ["-agentlib:jdwp=transport=dt_socket,server=y,suspend=n"]
  :extra-classpath-dirs ["/usr/lib/jvm/java-6-sun/lib/tools.jar"]

That's all you should need for OSX. For Linux you may also need to add something like this to point to your jdk's tools.jar file:

  :extra-classpath-dirs ["/usr/lib/jvm/java-6-sun/lib/tools.jar"]

Then run "lein deps" and "lein swank" and connect to emacs with M-x slime-connect.

Using Clojure Java source code

CDT supports using java source code in addition to Clojure source.

If you want to step through the Clojure java source, or see it as you traverse your stack frames, you will need to include the java source as a dev dependency. To do so, change your project.clj to include the following option:

  :dev-dependencies [[clojure-source "1.3.0"]]

Any other Java source you wish to step through can be included like so:

  :extra-classpath-dirs ["/path/to/random.jar"]

The clojure-source version should correspond to the version of clojure you are using, e.g: "1.2.1"

clojure-source is not currently available for versions above 1.3.0

Test Drive

Breakpoints/Stepping/Reval

From the repl, run:

(use 'clojure.set)
(difference #{1 2} #{2 3})
and confirm that it returns #{1}

Then set a breakpoint like so:

(use 'swank.cdt)
(set-bp clojure.set/difference)

Rerun the difference command; you'll see the source buffer pop up, as the bp is hit, along with the sldb buffer showing the stack trace.

Hit e for eval, and enter s1 to see its val, or any sexpr using the locals available in this stack frame.

Now put the cursor in the source file on the s1 and hit ^c^x^p, to see it's value, #{1 2}, on the mode line. Do it again with the cursor on the left paren in front of the (count s2).

Go back to the sldb buffer and hit x to go to the next line, "(reduce disj s1 s2)"; eval it and confirm that you see the #{1} below.

You can go up and down the stack frame by positioning the cursor on the stacktrace in the sldb buffer. Press v to bring up the source and confirm that you can eval things in different stack frames.

Hit q in the sldb buffer to continue execution; at the repl note the difference command finished.

Then run the difference command again ; when the source buffer pops up this time, set a line breakpoint, by putting the cursor on the "(reduce disj s1 s2)" line, and hit ^c^x^b. Continue and confirm that the second bp is hit.

Continue to exit, then rerun the difference command in the background like so:

(bg (difference #{1 2} #{2 3}))

Note that this leaves the repl free to execute other commands. For example, you can use the reval command to remotely eval an sexpr in the context of the frame last viewed.

(reval (count s1))

reval is the same function invoked by the e command from with the sldb buffer. Sometimes it's just more convenient to invoke from within the repl. (Plus that leaves you a history of the commands you've excuted.)

You can also view and delete the bps, like so:

(print-bps)
0 clojure.set:59
1 clojure.set/difference


(delete-bp clojure.set:59)
(delete-all-breakpoints)

Catching exceptions

From the repl, confirm that a bad multiply generates an exception:

(* 1 {1 2})
java.lang.ClassCastException: clojure.lang.PersistentArrayMap cannot be cast to java.lang.Number (NO_SOURCE_FILE:0)

Set a catchpoint:

(set-catch java.lang.ClassCastException)

Run the multiply again. Confirm that the source for clojure/lang/Numbers.java pops up.

Note that evaling the x and y parameters shows the 1 and {1 2} you passed into the multiply above.

Delete it when you are done, like so:

(delete-catch java.lang.ClassCastException)

Keystroke commands

These commands work from the sldb buffer

Keystroke Command Executed
s step into
x next/step over
o finish/step out
e eval in the context of the current frame
v view the source of the current frame
q quit the sldb buffer and continue

These commands should work on both Clojure and Java source buffers.

Keystroke Command Executed
C-c C-x C-b set Breakpoint, on the current line
C-c C-x C-g Go/continue, after deleting all bp's and catch-points
C-c C-x C-p Print/reval the sexpr/symbol under the cursor.

Other useful commands that can be run from the repl

Command Output
(reval <sexpr>) Remotely eval an arbitrarily complicated sexpr, from the context of the last viewed frame.
(bg <sexpr>) Run the sexpr in the background to leave the repl available for other debug commands

The Gory Details

Deadlocks

Any time a java thread is stopped with a debugger, it can cause deadlocks if it is holding a resource needed by other threads. If this happens, hit ^c^x^g to delete all bp's and catch points, and force the offending thread to continue.

I've noticed it happens most often if I inadvertently step into a java.lang.Classloader method. These methods hold locks which are needed to do any class loading, which swank itself often needs. Just hit ^c^x^g to continue.

Some locals in higher stack frames appear null due to Locals Clearing

One major weakness of CDT I've found is that sometimes valid non-null locals appear null. It's an unpleasant side-effect of the locals clearing the compiler does to reduce the danger of head-holding lazy seqs.

I've talked to Rich about this a bit, and I'm hoping for a workaround. Till then, if you go up or down the stack frame you can usually find other copies of the local that actually do show its correct value, particularly in java files where locals clearing is not done.

Clojure versions older than 1.3 can't reval locals in let bindings until the let body

Eval'ing a local will fail if the current line is still in the let binding that defines it. This is actually a bug in the Clojure compiler, which I have patched. That patch has been accepted in Clojure 1.3 Alpha 6. The same patch should work on Clojure 1.2 if you want to install it there.

If you are not using that, and want to see what the dependent locals are set to, you'll either have to step into their corresponding initializers, or just step all the way out of the bindings into the let body.

Reload resets breakpoints

The JDI sets breakpoints on methods; if you do anything to reload those methods, (eval'ing a require/use/defn), the breakpoints will remain on the old methods, which are no longer in use. They'll need to be reset by hand for now.

Editing file buffers changes line numbers!

This one is obvious, but it's so easy to do from within emacs, that I forget periodically, and you probably will too, and then wonder why the debugger is not going to the right line number.

Dynamic bindings are only correct in frame 0

reval is always invoked in the context of frame 0 on a suspended thread. The lexical scope for other frames is handled by pulling them out of the jdi and passing them into reval when it is invoked.

Because dynamic bindings are a clojure construct, the jdi doesn't know in which stackframe they get set, making it hard to determine how they should change across stack frames. I'm considering some fixes, but in the meantime, reval'ing a form that depends on a dynamic binding will always using the binding as it exists in frame 0.

Java 1.5 requires more work

Right now, swank-cdt only works on Java 1.6. I'll get 1.5 working if there is significant demand.

Still not sure how well it works on Windows

Thanks

Many thanks to Travis Vachon, Tavis Rudd, and Walter Van Der Laan for their assistance and encouragement. Also, thanks to Jeffrey Chu, and Phil Hagelberg for giving us THE tool.

Comments/Suggestions

Send any comments/suggestions to George Jahad at "george-clojure at blackbirdsystems.net" or to the main clojure mailing list.