Debugging Erlang programs 101

The first step in debugging Erlang programs is simple: Acknowledge that debugging concurrent programs is hard. Extremely hard. When you have that acknowledgement, the trick is to get your bag of debugging tools up to date. The very first thing you should do is to enable SASL. This can be done with the command

application:start(sasl).

or by using the boot-script that includes the SASL application. SASL will enable different kinds of information in the shell, but it can also be configured to output to a file. SASL will send reports in different situations. Whenever something CRASH-es or something PROGRESS-es, SASL will send a report to you. The next part of debugging is to understand how to dissect Erlang back-traces. Erlang will not report line-numbers where the error occurred. Hence, you should use many small functions, as the function will occur in the back-trace. You must also remember that the back-trace may does not include tail-calling stack-frames. How do you read a back-trace from Erlang? The first thing is to know what the different error messages means. The exit reason of a function will tell us what is wrong at the point where the error was raised. The next part of the stack-trace looks like:

** exited: {function_clause,[{foo,fact,[0.00000e+0]},
                             {foo,fact,1},
                             {erl_eval,do_apply,5},
                             {shell,exprs,6},
                             {shell,eval_loop,3}]} **

So what does this tell us? It says that the exit reason was a "function_clause" error. These are because no pattern matches in the foo modules fact function. The next term is a call to "foo:fact/1" and that comes from a call "erl_eval:do_apply/5" and so on. More evil are local "fun (...) ..." declarations which will be put into the stack as well. Watch out for these.

Assertions

The single assignment form of Erlang let us write assertions in a neat way in the code. If we set X = Y where both [X] and [Y] have already been defined, we define an assertion. Note this is also true if X or Y are (partially) constant expressions. This asserts that X and Y are equal. Use this to your advantage! Spray with assertions all over your code. Write down what you expect the value to be. Write functions that tests assumptions about the code. If the assertion is violated, you get an error right away. Use function guards. If you expect an integer, write

foo(X) when is_integer(X) -> ...

rather than foo(X) -> ... Sometimes, guards can't be used as only certain BIFs are allowed as guards. Then you, write a longer function that tests your assumptions and assert on that function. When things get concurrent, it is assertions that will save you. You should not worry too much about adding assertions to the code. Unless you add them in the cost-centre, it won't affect you much. Rather, you should worry about the correctness of the program. You can always remove assertions from critical parts and surround the critical part with a check if needed.

Dialyzer

One last debugging tool to mention is the dialyzer. It is a static checker for Erlang code and it is able to find many discrepancies in the code if you let it run on it. You should heed its warnings, for often they can alleviate a problem before it even becomes one. This concludes the 101 in Erlang debugging.

Books equals code?

Suppose you were an author writing books. A publisher wants to publish a novel on a particular subject; a sci-fi novel say. Now imagine that the author will be paid for each month he or she works on the book with a nice pay, but once the book has been written, then it belongs solely to the publisher. The publisher will sell the book for the next 20-30 years and make a bit of money off it each month. The author will not get anything of these money. A mediocre author would -- perhaps -- jump at this. A bestselling author, however, wouldn't. She would only accept writing the sci-fi novel should she get considerate percentage of the earnings. She would know that her value is built by accumulating it in the books she writes because they will throw off a small bit of money each month for 20-30 years. Now imagine you are a programmer at a company. Most programmers happily give over all rights on the code to the company. They happily get their pay each month, and when they get fired or move on, their accumulated value is lost entirely. The paradoxical thing is: most programmers accept this. Some of their worth is in the code they wrote, but since it is entombed in the repository of the company they have access to that worth in their next job. I command this to stop. Now. Here is a set of ideas: First, you could tie the success of the company with your earnings, but be sure that you get something other than salary. It is gone when you leave. Stock options is one way. Or you may be one of the owners. Second, you could ignore the company and write code "in stealth" on the side of the company. When you have a product you could try to shrink-wrap it into something salable. With luck, you can then either get something fun out of it, or a lot of money. Note that many companies disallow such activity. If you are at one such company, I suggest you move on and let them outsource your seat to India. They will be sorry for trying to outsource the system kernel (it always fails). Third, you could have the company release part of their code as open source. This leaves you with a way to take your work with you to another company and with luck you also generate "street credit" in the Open Source community. In other words, you are generating accumulating value you can take with you when you leave. The company can get a feedback loop going on such open sourced software, so it might end up being beneficial to them. Fourth, you can write software and release the software under an open source license. But you only do work on the system by contract. If a company wants tailoring, it is negotiable for a price. They get the best programmer for the job and you back-fit the changes into your code generating even more value in the project. Another path is to take money for support on the code base.