In case you have any doubt, I advocated the commando approach,
fighting the good fight for programmers to be able to do their own thing
If you think of coding in military terms, there is an exactness to the phrase
code commando. There is a reason Thielen did not style himself a
code jarhead. Commandos work alone or in small teams; they are highly
skilled; they do dangerous missions where backup is unavailable; and in many
cases they, not their commanders or comrades, take ultimate responsibility
I was wrong and Larry was right. (Boy, that hurt!) -- David Thielen
Commandos can do what grunts do if they have to, but they don't like it; they
are better than grunts, they know it, and they act like they know it. (If
you've ever known a bona fide Green Beret or SEAL, you know what I'm talking
about.) Grunts, whether alone or in arbitrarily large groups, cannot do what
commandos can do. That's why the military goes to great expense to create
such people, even though they are a pain in the ass to work with.
Code Commando Boot Camp
In the military, commandos go through special training. Code commandos will
be quick to tell you that the usual educational channels will no more make
you a code commando than standard boot camp will make you a SEAL.
The code equivalent of commando training usually involves a primitive computer,
a daunting task, and a large amount of unpaid time. The task may involve
reverse-engineering a large amount of undocumented code, or fitting 27K of
application into 16K of RAM. The budding commando immerses himself not
only in the machine and task at hand, but in every manner of skill even
remotely connected with it. He tries many things that do not work. One day,
if he does not wash out and give up, he will receive the epiphany by which
all code commandos come to first awareness:
Computers really aren't all that complicated.
By corollary, there is no such thing as a "hard" language, since all computer
languages are more or less the same (give or take a few minor strengths and
weaknesses). There is no such thing as a large task, if you break it down
into fine enough pieces.
It is hard to overestimate the importance of this. The code commando doesn't
understand this as a theoretical platitude; he has lived the truth of it.
It's easy enough to assert that, with a modest sustained effort, you can walk
from Los Angeles to New York; it's quite a different thing to have done it
with your own actual feet. The code commando has walked the walk and he feels
an easy (and sometimes ill-considered) contempt for the code wussies who have
only watched it on TV.
Code Commando Limitations
Commandos can't do everything. The team that can blow up a bridge behind
enemy lines cannot organize something like the D-Day Normandy invasion.
This seems to be the basis of Thielen's reverse epiphany. "Modern projects"
(his phrase) are now too big for the commando approach, and techniques are
now well-formed enough that the commando's ability at field improvisation
isn't necessary. The commando's strength becomes a liability, since the
on-the-fly changes he makes to his part of a project sabotage his neighbor,
who is relying on the API to work as specified for his code to work.
This is a legitimate point, one Thielen makes with an air of apology. Yet
he also makes several points in favor of the commando approach:
What is going on here, it seems, is that Thielen himself has gone up the
ranks; once a successful commando, he now wears a general's bars and has been
asked to organize D-Day style invasions. And yep, he's found that you can't
use commandos for that, any more than you eat soup with a fork. You have to
use grunts, and if you have to write the directions for them in comic book
form, then you deal with it.
- "I still believe the best team size is two or three people."
- "Nothing can make up for having poor programmers."
- "I have yet to see CASE tools do anything other than slow down a project."
- "It is also crucial that you give programmers ownership of the pieces on
which they are working."
- "Closing off windows, requiring set hours, enforcing a dress code, and
supporting other stupid human resource edicts can also contribute to lower productivity."
The hidden assumption in this is that all programming now consists of
these million-LOC super-projects and that the commando has outlived his
usefulness. I would dispute this on several fronts.
The Myth of the Small Project Demise
I have worked for the same company continuously since about 1985. For most
of that time I have been programming; at first as a sideline to my "real"
job as a service technician, then for the last 13 years or so full-time.
In that time I have completed over 2,000 separate projects. Most of these
have been sub-1KLOC quick hacks. There is a constant demand for this work
because there aren't many people who can do it right. A lot of
little industrial devices come with their own pidgin control languages, but
not many people can write programs for them which work reliably and (more
importantly) fail gracefully when something goes wrong.
At the same time I've done some projects that are very large, to a one-man
shop such as myself. I've done front ends liaisoning with customer mainframes,
including several AS400's and a couple of enterprise-level SQL databases.
I've done real-time systems doing plantwide data collection. I've done
high-speed classification and complex statistical reporting. And I've done
it all in the classic commando form. I have delivered several projects
very late, I will admit, but I can count the projects for which we
weren't paid on the fingers of one hand. And I have had exactly one project
which failed to work, after management insisted I use a piece of hardware
I warned them was inappropriate. That was in 1987. Fortunately, our choices
aren't as limited today.
If the small project was dead, I would be on the unemployment line instead
of quoting deliveries in 2004 for people who are willing to wait.
What I have often found is that the small-project front end is more reliable
than the big-project database it's fronting for. The myth is that my kind
of work is being obsoleted by "everything" systems like SAP which integrate everything
from the shop-floor user interfaces to the drill press controls and the
accounting database into one overarching package. The truth is that the
end-user companies who try to implement such systems keep coming back to
me for bits and pieces they know will work.
If the code commando isn't an appropriate soldier for the big project, then
it must also be said the big project isn't an appropriate tool for many
real-world problems. The big project has inherent limitations; it is never
as reliable, the user interface is never as smooth, and it is much harder
to update when a problem surfaces. These aren't reasons to retire the code
commando, though; they're reasons to question the deployment of big projects
in places where they aren't appropriate.
The Sloppy Commando
Because the training for code commandos isn't formalized, we tend to pick
up a lot of bad habits along with the First Epiphany. We develop language
and platform chauvanisms, and fail to document things we consider "obvious."
Code commandos who have these faults can still accomplish amazing things,
but it's correct to criticize their shortcomings.
Code commandos who actually work in commando style generally reach the
Second Epiphany within a few years:
YOU ARE FALLIBLE. You must take certain precautions because of this.
It's the military equivalent of being asked to lay out a minefield, then
to pick your way through it a year later; not too easy if you've forgotten
where the mines are. If the code commando's style is forged in the cauldron
of an impossible project tackled and conquered, it is tempered with the stink
of failure. One day the budding commando looks back over a chunk of code
written a month ago and scratches his head -- What fool wrote this crap?
Oh wait, I wrote it! Or he utterly fails to follow the "obvious"
logic which he didn't bother to document. One way or the other, he realizes
that he must comment his code and follow some conventions so that he and
others can understand it later.
Writing really bad code is an occupational hazard of being a commando;
what Thielen seems to have forgotten is that it's a necessary part
of maturing into your vocation. If owning your own code gives you a heady
sense of independence, you must also be prepared for the personal stigma
of failure when that code fucks up. You can't lay it off on the team or
the manager. It's yours, buddy. You get to fix it and wipe up the mess
while everyone looks at you.
Back in 1989 I wrote a program with a bug in it. (Hard to believe, huh? :-)
What happened was that the cover of a 20-ton baling machine closed while
a sheath used to guide falling product was still in the "down" position.
This resulted in two very expensive pieces of metal being forced to
occupy the same physical space by a ram strong enough to crush a car.
The end result was that I had to watch for an hour as five very tough
plant workers whaled away on the sheath with sledgehammers to bend it
back into shape. All because I made a mistake.
Things like that are why I am just a little less tolerant of
sloppy code than most people.
Thielen says this about the code commando:
Without up-front architecture and design, you are almost
guaranteed to write code that, as you progress, cannot do what you
need it to do.
That's very true. But it's not a limitation; the code commando gets used
to rewriting code. You get used to doing whatever it takes to make it
work. That takes time and, often, rewriting. The problem is that rewriting is
bad in the context of a large project, because it interferes with
what the other coders are doing.
The Myth of the Big Project
There is no such thing as a million-LOC project, or even a thousand-LOC
project. If you have something that looks like a single task with this
kind of line count, what you have is a bad specification, which you should
change. But in a big project, you can't do that...
A perfect example is the Windows API. It is chock full of really bad
specifications which couldn't be changed once written, so they are only
half-implemented or worse implemented plain wrong. There is no mechanism
for some guy in a cubicle at Microsoft to say "Now that I'me coding this
function, I realize three of the seven specified error messages are almost
never used and one of the arguments is both unnecessary and a pain in the
ass to deal with." He can't do that because six other wonks in cubicles
are coding other functions which use the API whose deficiencies he has
What seems to happen at Microsoft is that this guy calls the other six
wonks and says "Do you really need _____ to be implemented?" and if there
is a consensus, he quietly omits to support the API as written. This
is the worst of all possible worlds, for obvious reasons. It's also
inevitable when you over-specify projects from the top down.
The Windows API is plagued by too many error messages which must be
laboriously checked, too many arbitrary command parameters which may
or may not be supported in real code, and too many conflicting and
arbitrarily chosen ways of doing the same thing. There is nobody in
a position to say "We're going to decide once and for all when to use
variants and when to use strings, and change everything to make it
consistent." Contrary to popular belief, the Windows API is not so huge
it can't be understood by a single person; it's only a few dozen pages
and there are a modest number of functional groups. It's only such a
pain to deal with because there are so many exceptions, so many rules
While it's true you need specifications to hold a large project together,
you shouldn't need to slavishly follow specifications which, at coding
time, can be shown to be stupid. This is why you code from the
bottom up, not the top down, and you build your more abstract specs from
proven building blocks.
I have never seen a project so large it could not be broken into small
chunks -- say, 100 lines of code max. However, in the process of
subdividing a project you often run into broken specifications which
can't be handled without byzantine exception handling and other tricks
which keep your chunks large. The code commando would rewrite these
broken specifications, but the coder on a large project can't. This
is not a problem with code commandos; it is a problem with the handling
of large projects.
In the original discussion, I made this comment:
If you make your bricks strong enough, you don't need to
worry about the architecture of your dome.
I stand by this comment. If you build a dome of masonry, you need to
heed the loading of your bricks and mortar, and build it carefully so
it's supported during construction. But if you weld your dome together
out of steel plate, or fire it as a monolithic ceramic construction,
you can lift it up bodily and turn it upside-down without compromising
its integrity. You don't have to pay close attention to its exact
shape, and you are freer to indulge in artistic whimsy.
It's the same with code. Make your blocks small enough, strong enough,
and solid enough, and the larger blocks you make with them will inherit their
strength. You can do this from the bottom up through as many layers of
abstraction as you need, and build arbitrarily large applications that
will not crash or fail no matter what you do to them.
When you do this you can't dictate the exact shape of the final product
at the outset, though, because you may not know what kind of bricks
you'll end up with in the high layers of abstraction. This kind of
programming involves as much exploration and discovery as it does
imposition of order. You can work toward a final goal, but you may not
end up with exactly what you expected. You can only be sure you
will end up with something that will do what it can without falling down.
The Myth that it's Not Important
Industry is different from Web design, so I'm occasionally reminded. My
code can destroy hardware and kill people. You don't have to be as careful
in other venues, where the economics are more important.
To which I say: tell that to the people who use that other software.
Large sloppy applications like Windows and SAP are increasingly found doing
things that are life-threatening and mission-critical, such as
controlling emergency dispatch systems, automatic teller machines, and
maintaining medical databases. (see just about any issue of the Risks Digest.)And as a user I resent the hell out of
software which assumes I can be bothered to periodically reformat my hard
drive or be denied access to services because the authors couldn't be
to do their jobs right. When I talk to non-programmers about this they
always agree; it's not that they don't care, it's that they don't realize
there is an alternative.
Customers want software that works. When they get it, they like it better
than software with more features, snazzy graphics, and automatic
background functions. It doesn't matter how "smart" the application is
if it's down half the time, or if you can't depend on it doing what you've
told it to do.
In my career I have often delivered programs that differed markedly from
what was discussed at design time. I never provide screen shots, since
I tend to tweak the layout of user controls right up to the last minute.
I have omitted functionality which I felt compromised my software, but
more often I've added functionality not asked for because it was so easy.
I have never had a complaint because of this. Quite the opposite; I'm
sought out precisely because I'm willing to go the extra mile to make
a system that's robust and easy to use.
I have called customers about mainframe interface API's and asked to
rearrange data; I've never been refused. End users don't care whether
a system exactly matches the specification; they care, very deeply,
whether it works. Often a form layout or tab-order makes the
difference between a system that is a pain in the ass and one that makes
the work day breeze by. I like to tell customers that "the hard part of
my job is making your job easy." If a customer has cause to notice the
quality of my work at all, it probably means I've done something wrong.
The best software is invisible to its users.
Like the other kind of commando, I'm most successful when you never
realize I was there.