I was asking my self the same thing around half year ago: "Wow, it's Knuth here and he is saying such strong words!" I then decided to give literate programming, incarnated in his CWEB, a try.
I have written two free programs using Knuth's CWEB:
- Real-time telnet game with ASCII animations where each player has a ship and is supposed to destroy others.
- Internet super-server. Actually, the name is wrong in that it isn't connected in any way to email spam. Basically, a program to send as ``requests'' (using whatever TCP/IP protocol: it could be HTTP, SMTP, XWindows, whatever) to a given internet daemon as fast as the underlying operating system allows.
In the first, I didn't use CWEB rightly. I didn't take the time to write comments properly. And I don't regret that. CWEB was, however, useful because of the macro systems it provides.
In SpamBot things were differently. I took a lot of time to actually write down what I was doing. You can see the resulting documentation as PostScript (268 KB) and as compressed PostScript (68 KB).
I have not received any code contributions at all for any of those programs but, on the other hand, neither have I received any code contributions for many other free programs I have released.
There is something I would change in CWEB: I would make it possible to specify parameters to the @<macros@>. In his papers, Knuth says he thinks this is not required since you already have C's #define macros (available in CWEB using the @d construct). Yes, that's right, there are, and you could live using nothing but C's #defines. Eventually, you'll run into two problems.
The first is that you can't call @<macros@> inside of them. That's okay, you don't need @<macros@> but just @d macros. Fixed. Just remove the @<macros@> from CWEB, they are not needed.
The second problem, however, is that __LINE__ is defined ``wrong'' inside of C #defines (at least when compiling with GCC). I use ASSERTs heavily in most of my code. If you take a look at Matanza or HB, you will see there are functions with far more lines with ASSERTs than actual code. When one of those ASSERTs is triggered, the __LINE__ where the error took place is reported to be the line where the definition was used, not the actual line inside the definition where the __LINE__ was written. This may not seem that big a problem, but when you have #defines that use other #defines that use other #defines that use other #defines, this is a very big problem: there is no easy way for you to know which was the line on which the error happened.
A priori, this would seem easy to fix: just make it so the compiler will report the real line where __LINE__ was written, but that would make things even worse as all those __LINE__s were written in the definition of my ASSERT macro! So whenever an error takes place, no matter where (in a @<macro@> or a @d (#define)), it would point me to the definition of my ASSERT macro.
But Knuth is not interested in my opinions on CWEB anyway.
Why haven't I ``fixed'' CWEB to have its @<macros@> accept parameters? Even though it would make my life easier on this sense, I would introduce yet another incompatible tool to make literate programming. People wanting to study my software would need to learn it, not standard CWEB, to be able to understand/modify it. They would also have to install my own version, not standard CWEB, to be able to test their changes. So I'm still sticking to standard CWEB.
I know there are many other alternatives to CWEB. I guess some of them fix this concern I have. Could anyone suggest one? I don't want to spend way too many time looking at different alternatives.
I think that rather than TeX, a literate programming system ought to use HTML (or perhaps XML? Umm) these days. The cweave program ought to spit a set of HTML files for every macro (or a CGI script in Perl or CGI application in C or Java servlet or HB code or PHP code or whatever). Each HTML file would have the following information:
- Comments in HTML.
- Code of the macro/section.
- Links to all the other sections that use this one.
- Links to all the sections used by this one.
So why not take this one step further and design an entire IDE based on the browser? One would make it possible to modify the code and comments using HTML forms. The IDE would keep versioning information for both the code and the comments (who changed what when and why). I would even add a way to make assertions that can be incorporated as code at the beginning and end of the sections/macros to make programming by contract easier. And, of course, I would add useful views such as the whole list of sections, the sections last modified by a given programmer, a tree of sections and so on. The IDE would also require locking a section, to make it possible for any given number of persons to work on the code simultaneously. Finally, it should be able to export the final code in any number of compressed formats. Is anyone interested in codeveloping said system?
In my experience, literate programming does make programming more enjoyable.
One problem literate programming has is that rewriting the software means rewriting the documentation. This might be not that relevant in some contexts but in others certainly is. When you are talking about a program on which you'll be throwing big portions of your code away every now and then, rewriting the documentation adds a significative burden on the programmer. There are times when changing a single line of code requires you to change three paragraphs explaining what you are doing.
Basically, I recommend literate programming for software that you don't think will be changing much in the future. Examples of this are TeX and CWeb, which have not changed in a long time. Some persons argue that all the software ought to be written that way: so one can talk about whether a program is finished or not. That discussion is outside of the scope of this comment. In other cases, in software that is continually evolving, I don't recommend literate programming that much. But when you have clearly defined what your program will and won't do, in a way such that thinking about future versions (of the program itself, this does not include packages that include the program combined with others: you can think of a program as a module of a bigger application) makes no sense, I recommend the use of literate programming very much.