Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
[Un] Dependable software

By Maniac in Technology
Fri Dec 08, 2000 at 02:50:22 PM EST
Tags: Software (all tags)
Software

There has been several discussions about the quality (or lack thereof) of software. In my mind I think of "best value for the customer" and am looking for feedback on that concept and how it affects how we build systems that use software.


ADVERTISEMENT
Sponsor: rusty
This space intentionally left blank
...because it's waiting for your ad. So why are you still reading this? Come on, get going. Read the story, and then get an ad. Alright stop it. I'm not going to say anything else. Now you're just being silly. STOP LOOKING AT ME! I'm done!
comments (24)
active | buy ad
ADVERTISEMENT
I was pointed to this article at Salon titled "High tech's missionaries of sloppiness" from a mailing list. It goes on at quite some length but basically compares the US software industry with the US Automobile industry of the 1950's. In some ways, this isn't "news". Ed Yourdon in 1992 wrote "Decline and Fall of the American Programmer" (ISBN 0-13-203670-3) which made three main points:
  • Software professionals are far less expensive in many parts of the world than they are in North America.
  • Software professionals in many parts of the world are substantially more productive than the average software person in North America.
  • Software professionals in many parts of the world can develop software with much higher levels of quality than the average software produced in North America.
These were reexamined in Yourdon's 1996 book titled "Rise and Resurrection of the American Programmer" (ISBN 0-13-121831-X). In that book, the comments point more towards new areas of expertise and other factors in the continued prosperity of US software companies. There continues to be a lot of discussion about India's software industry and other factors, but I believe these all miss the point.

My point is this - a customer is looking for "value". I define "value" as the difference between the benefits they receive from a product or service and costs. Value can be measured in several ways - cash, reduced time to market, improved accuracy, "being hip", and so on. The fact that a software product has defects (bugs is too polite a term) is just one factor in determining the value.

With this context in mind, what I ask developers to do is...

  • manage the size of the product (yes - lines of code) since that affects development costs, computer resource usage, etc.
  • check their product using "their customer's" criteria prior to hand off. Do you think it is defect free at that point?
  • manage conflicting goals - on time, under budget, zero defects. I want them all. The developer must consider all three when deciding what to do (and which to violate).
  • deliver a simple system early that meets the basic need and improve it based on usage.
Is this a better way of looking at software development than a "no defects" or "improve productivity" view of development?

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Poll
What is most important to your customers?
o Defect free 30%
o Capability 36%
o Cost 5%
o Security 4%
o Don't know 7%
o Don't care 15%

Votes: 69
Results | Other Polls

Related Links
o Salon
o "High tech's missionaries of sloppiness"
o Also by Maniac


Display: Sort:
[Un] Dependable software | 21 comments (21 topical, editorial, 0 hidden)
Poll (3.50 / 6) (#1)
by spaceghoti on Fri Dec 08, 2000 at 01:30:10 PM EST

What, no multiple choice?

My customers are internal employees of my company: my department works to develop software solutions for other departments. What they want are speed, reliability and cost in no particular order. However, since I couldn't pick all of them (and speed wasn't there at all) I went with "I don't care." Personally, I just make sure the tools the developers use work properly. But they still want my services quickly, reliably and cheaply.

I'm seeing a trend here...



"Humor. It is a difficult concept. It is not logical." -Saavik, ST: Wrath of Khan

Mostly right (3.28 / 7) (#2)
by maketo on Fri Dec 08, 2000 at 01:40:12 PM EST

Many a recommendation you give at the end of the article, if not all are good,yet unimplementable. The problem is that there is simply to much money on the market. Even if you come up with a crappy product - you still get paid. Right now there is no real incentive to do testing, education, qa, real design and planning,lot of prototyping etc. In this way even the free software outhere does not contribute to the quality... Take a look at many a person hired to do coding work and/or anything else -> some of them get hired as low as 15 years old. While I dont have anything against the youngsters - many of them are simply kids sitting on irc 24/7 without any _real_ skills, just a lot of _user_ side experience.
agents, bugs, nanites....see the connection?
Definitely. (4.00 / 4) (#3)
by porkchop_d_clown on Fri Dec 08, 2000 at 01:55:12 PM EST

It's becoming incredibly hard to tell the real programmers from people with a degree or certification in programming.

Unlike almost any other profession, programming depends on the programmer's ability to think clearly and logically. I'm almost always disappointed. There are more programmers everyday, but it seems as if the number of people who can actually think like the computer are a constant.



People who think "clown" is an insult have never met any.
[ Parent ]
I blame the customers (4.55 / 9) (#4)
by rusty on Fri Dec 08, 2000 at 02:10:09 PM EST

Ok, I know how that sounds, but listen for a minute.

Here's what people want out of software:

  • It has to be cheap
  • It has to have every possible feature
  • It has to be completely bug-free
  • [In the case of code-for-hire] It has to be done yesterday
In other words, people want perfection. Perfection, in software, is not acheivable. In my experience, the cost-- in features, in time, and in money, increases exponentially the closer you try to approximate perfection. If I had an infinite amount of time and money, I could give you perfect software. Any less than that, and you start having to balance imperfections against each other.

So, if you have to give some leeway on some of the above, what order do they come in?

People want bug-free software, yes. But, give them a choice between something that is available *now*, but might be less-than-bug-free, and who waits for someone else to release a more stable version?

People want bug free software, but if you release something that works perfectly, but only has 20% of the features as this other (more buggy) app, they'll choose the one with the features.

People want bug-free software, but if you offer them two programs, and one costs twice as much as the other (for the same featureset), which one do you think they'll choose?

The point is that currently, there is basically no market for bug-free software. If you ask people "Have you been exasperated at the bugginess of your software lately?", of course they will say they have. But consumers create the market, and the market demands Availability, then Features, then Cost, then Stability, in that order.

So, developers know that someone will be unhappy about something. What it comes down to is that people will suffer with buggy code in exchange for features, low cost, and timeliness. They complain more if you focus on stability at the expense of any of the other factors.

So, until someone comes up with the magic bullet that makes bug-free code able to compete economically with buggy code, it's gonna stay this way.

The point made in that salon article about the AK-47 is very apropos here. The AK-47 is a very simple machine. Users of automatic rifles appear to realize that they need them to not jam, much more than they need lots of bells and whistles. Here is a case where consumer demand favors a simpler system with less features. I hope and pray that people will realize, eventually, that an application should do what it does simply and well, and that every app doesn't need to expand until it reads email. But until that happens, bad software is here to stay.

____
Not the real rusty

What if you held companies responsible. (3.20 / 5) (#10)
by amokscience on Fri Dec 08, 2000 at 03:20:03 PM EST

In an effort to prolong the discussion...

How about we hold software companies accountable for the quality of software. (Does this get into UCITA?) Obviously this has huge implications both ways...

Where do you draw the line?
Who maintains standards?
What about 'garage developers'/Open Source?
What about excessive lawsuits?
How do you keep big money out of the process?
What about International standards?

Do any countries have any such laws?

I'm sure enforcing stndards would be a huge showstopper for most software companies. Engineers deal with it in their industry and programming is slowly moving towards more and more engineering type methods and standards. I'm a programmer myself with a strong interest in processes and I can see how it would be a push in the right direction, but probably excessive. And I wouldn't want it to stop me from making my $20 or free programs.

[ Parent ]
Hmm (4.33 / 6) (#11)
by rusty on Fri Dec 08, 2000 at 03:34:50 PM EST

As for enforced standards, there are some cases where that matters. In cases where software is life-or-death, yes there should be strictly enforced standards.

OTOH, does it matter if my FTP app conforms to USG Code Regulation 834-X? I doubt it. In the case of the majority of consuimer software I think there's no overriding reason for there to be mandated standards.

____
Not the real rusty
[ Parent ]

Not mandatory (3.00 / 2) (#19)
by Eimi on Sat Dec 09, 2000 at 12:42:57 PM EST

I think having mandatory standards would be a really bad idea. The problems are several and you touch on most of them, but I think the real problem is that large companies could afford to mee the standards (which after a few years of drift might have no relation to actual software quality), but there's no way free software could compete. (I mean, come on, Windows NT is POSIX compliant and Linux isn't. Say what you want about the relative quality of them, it's pretty obvious which is closer to the point of of the spec.)

What might work, though, is optional standards or guarantees. Particularly in fields where software is expensive and risks are high, a real in paper guarantee that it will work or else might be a huge benefit.

[ Parent ]

I blame the system (4.50 / 2) (#13)
by kagaku_ninja on Fri Dec 08, 2000 at 06:02:00 PM EST

Start-up company comes up with their new widget idea, develop demos and pitches the idea to investors. VCs fund them with $20 million. A year later, the money is running out, and the board wants to see results...

Another problem is feature creep, which comes from industry analysts who produce "check-box" comparison lists. If you don't implement a feature your competitors have, customers are going to think twice about buying your product, even if they may not actually need the feature. This encourages management to insist on developing lame implementation of said features.

If your product is excessively buggy or slow, you will get killed in reviews. However, if it is "good enough", then your company can hopefully survive and fix it "later".

Once you come to dominate a market, or have users locked into using your product, there is little incentive to fix bugs. More likely, management will want new features, to create reasons for existing users to upgrade.

[ Parent ]
Vicious cycle (4.50 / 2) (#20)
by rusty on Sat Dec 09, 2000 at 08:13:14 PM EST

It's all a feedback loop. Your examples are the responses of the system to the consumers, and the consumers expectations are reinforced and fed by these actions on the part of the system. Both sides are to blame, but how can the circle break? I think it's more likely to happen by software buyers standing up and demanding quality instead of feature checklists. But you're right, both sides play into it.

____
Not the real rusty
[ Parent ]
Good, Fast, Cheap. Choose any two. (3.37 / 8) (#5)
by greyrat on Fri Dec 08, 2000 at 02:28:16 PM EST

Good, Fast, Cheap. Choose any two.

'nuf said.
~ ~ ~
Did I actually read the article? No. No I didn't.
"Watch out for me nobbystyles, Gromit!"

Foreign Workers Better? (3.50 / 8) (#6)
by jack_richins on Fri Dec 08, 2000 at 02:37:59 PM EST

Perhaps I just haven't worked with the right foreign programmers, but the ones I have worked with have produced much buggier code than what I see from my American associates.

Anyone that has read Yourdon's book, how does he back up his claim that foreigners produce more bug-free code than Americans? It's not that I don't think foreigners could get to that point, but usually when they do they end up working for an American firm. I tend to think American Software firms are less picky about where they get their employees than American auto companies were in the 50's and that tends to counteract the improvement in quality of foreign programmers.

maybe it's from SEI (4.00 / 4) (#9)
by amokscience on Fri Dec 08, 2000 at 03:15:06 PM EST

I think some of those assertions come from the fact that a majority of the highly rated SEI level 4/5 programming groups are non-US (India has a large concentration). However, I too haven't run into too many foreigners that are any better than homebred Americans. In fact, I'm reverse engineering/rewriting some code that a foreigner wrote that's an incredibly intricate mush of C and C++ with absolutley no documentation or reasonable design choices.

That said, most foreign workers I've met are harder workers than their US counterparts... and most seem to have a genuine interests in their work.

[ Parent ]
SEI (4.50 / 2) (#12)
by kagaku_ninja on Fri Dec 08, 2000 at 05:19:01 PM EST

I currently work for an ISO 9001 certified software company. While the company does have a process for development, I haven't seen any evidence that the code produced is of higher quality than that at my previous job (which would rate a 1 on the SEI maturity level). In fact, this company blows deadlines, and operates under crisis-mode management, much like the last one.

Of course, ISO 9001 is not SEI; I don't claim to know much about either... My experience is that what really matters is the quality of your developers and managers.

[ Parent ]
Processes and levels (5.00 / 2) (#21)
by amokscience on Sun Dec 10, 2000 at 10:56:24 PM EST

I agree, the developers make a difference, but so do the processes. Merely having a process isn't enough. The SEI 4/5 groups are constantly seeking to improve their processes. They haven't implemented a single 'magical process that results in zero defects' (No silver bullets). However, the corresponding product developed as a result of the processes usually results in extremely low defect ratios. It's little surprise that many of these groups deal with aerospace industry software. They check their processes, analyze why something went wrong, and try to fix it. Companies running in crisis mode never stop to do that, or only do that as a token gesture to maintain that SEI 1 rating.

Profiles of SEI 4/5 groups reveal that they are anything but chaotic. They thrive on being stable, steady, and boring. I've never had the opportunity to work at such an organization (I might hate it, who knows). The highest I've been around is SEI 2 and it looked like a pretty placid place to work.

You can have excellent software come out of SEI 0 groups but the higher the level the more likely it is that the software will adhere to it's design specifications and have a low defect ratio. I'd claim that *if* you have processes then it depends on how strongly your managers and developers buy into the process.

One last thing. Some note that most of the SEI 4/5 groups seem to have 'ideal' development conditions in that they have 'get it right' as a priority over delivery date. Most of them have a mandate for a zero defect product (Mmm, $150 million satellites going *zap*) and large resources. That situation certainly doesn't apply to most of the worlds software shops. Personally, I'm interested in where the middle ground is.

[ Parent ]
People expect computers to be unreliable. (4.00 / 7) (#7)
by nogin on Fri Dec 08, 2000 at 02:54:59 PM EST

I believe that software industry managed to convince most computer users that computers are inherently unreliable. They do not understand that it's just a question of quality, they believe that it's just the way computers are. Because of these beliefs, they do not mind buying buggy software from the same company over and over again.

As soon as computer users realize that it's possible for their computer systems to work reliably, they will start demanding quality softawre. And we shouldn't just wait for that to happen on it's own - we should try to educate at least our friends and relatives. Explain to them that having to deal with viruses, to run mantenance programs to remove random junk from your computer and to do other things that Windows "power users" have to spend hours on is not a necessary part of using a computer, that they only have to do all of that because they are using <u>very low quality software</u>.

I find the opposite to be true (4.50 / 4) (#8)
by trust_no_one on Fri Dec 08, 2000 at 03:11:50 PM EST

Actually, I find that my expectations, as a computer professional are much lower than those of the average user. The difference between the professional and the amateur user is that the amateur expects the computer to work; the professional knows better. My family and friends are continually surprised when their software doesn't behave as expected. I have come to take such things as a matter of course. I am surprised when a piece of software actually performs as advertised.

As a developer, I have mixed emotions about this. On the one hand, I know well the constraints most developers are under, in terms of time and resources. On the other hand, I do get really frustrated when I think about some of the fatal bugs I've encountered that I know should never have made it into a shipping version.

I think software is still in the early stages, roughly comparable to the automobile in the early 20th century. It works, most of the time, but it requires a lot of maintenance and hand holding and you can't quite shoot your horse yet.

--
I used to be disgusted, now I try to be amused
[ Parent ]

Tools (4.42 / 7) (#14)
by StrontiumDog on Fri Dec 08, 2000 at 06:10:20 PM EST

In addition to the other points raised, here are a few:
Software quality is partly the way it is because of the tools used.

The programming language: Useful as C and C++ have been, in my experience half of all bugs in C/C++ programs I have seen have been solely due to language features: pointers, array bounds overflows and memory leaks being by far the biggest culprits. Yes, there are third party tools to help catch these kind of errors, but they don't often get bought and are even more rarely used properly. What I would like to see in a language: eliminate pointers, have arrays check their bounds, and include automatic garbage collection with programmer-controllable activation. We have Java, which would have been perfect except for the compulsory object orientation, the stupid VM, and the irritating garbage collection behaviour: just when you don't want it to.

Programming paradigm: The other half of most bugs comes from the program structure itself: a program does not quite behave the way the programmer intended to or expected it to. Sometimes this happens because of a logical error; mostly it happens by starting with a clean simple structure, and then adding features without much forethought until the structure breaks. This can be due to project management inefficiencies: adding features after design time is over. This can be due to the language used: I find that C++ does not encourage good class reuse simply because available base classes never quite do what you want, or do things in ways you never expected.

Hype above substance: Many programs are developed on the basis not of user needs, but of marketing hype. Windows, and of late, Linux, are very guilty of this. The contradictory aims of writing programs that can do everything except walk the dog and fix dinner, and yet remain simple enough for my half-blind granmaw to use, lead to monstrous programs with wierd and sporadically unpredictable behaviour. Programs that install themselves with one click, but are impossible to remove without editing the registry. Programs that do not clean up after themselves. Programs that pretend to be simple clickety-click browsers but turn out to also be email clients, news readers, telnet, ftp, LDAP, and WebDav clients, Java Virtual Machines, thingies that need Flash and RealPlayer and Comet Cursor etc etc etc, with interfaces for all these components wrapped up in one granmaw-friendly user interface, and a steaming mess of a lot less programmer-friendly APIs and COM hooks.

There are more, and I would dearly love to go into a serious language rant, but I guess these are three important technical factors in software quality.

Some quality management (4.75 / 4) (#15)
by slaytanic killer on Fri Dec 08, 2000 at 07:16:42 PM EST

Some thoughts:

. Software reliability comes from mindless programming. The less thought there is, the more mechanical and error-free the system. Under a certain smallness, this falls down, but in general you want regularity, less gotchas, less tactical thought needed for implementation. Let the genius be in the design.

. Therefore use design patterns and label them as such. It cuts down on the need for commenting (just say "observer design pattern" and it will be clear). Have a few copies of the book lying around. People will be curious why there is such an abundance and take a look at them.

. Also, make sure you have very stable development systems. Otherwise the developers won't have any idea if the bugs are in their code or in the system, and will start blaming the system even if it's their fault.

. It is in general impossible to mandate "quality." Of course everyone wants some level of quality. But quality is a process, as demonstrated by Six Sigma ideas, and you want to optimize the environment to produce quality.

Those are some thoughts; reading the others' posts, the only things I would say is that ISO quality metrics are overblown and thoughtless. And it likely doesn't help to say "You need more quality." No one wants to develop with bad quality; bad quality is already demoralizing. What is needed is an environment where quality precipitates.

Careful (5.00 / 2) (#16)
by kraant on Sat Dec 09, 2000 at 07:48:57 AM EST

Using patterns is generaly a good idea but documenting your program using them and only them would be rather counter-productive since they tend to be generative and therefore the aim of the pattern would not be immediately obvious.

Also keep in mind that not only are patterns one of many competing design philosphies available to the coder/designer but also that depending on who you talk to the patterns themselves differ. Keep in mind that they were originaly advanced as an attempt to codify beauty and aesthetics in architecture and therefore the patterns themselves can't be viewed as anything more concrete than a rule of thumb.

If you realy want to make sure the code is good, clear documentation is an essential as is proving and regression testing it.

In the real world no-one is willing to pay for all this more's the pity. But I just wanted to make it clear that Patterns although useful aren't a Silver Bullet any more than Extreme Programming or UML are.


--
"kraant, open source guru" -- tumeric
Never In Our Names...
[ Parent ]
Agreed, with explanation (5.00 / 3) (#17)
by slaytanic killer on Sat Dec 09, 2000 at 09:51:03 AM EST

While writing my post, I stared at that point for a moment. But for something that is documented as the Observer design pattern, all that you need in addition is a short explanation of the API and the normal internal code comments.

Sometimes you want to emphasize the black-box nature of something by minimizing the API comments. That does not mean you can decrease the internal code comments; the scale is too small to use design patterns.

That said, commenting is extremely important; I am coming from the position that you care enough about commenting that you have the luxury of being minimalistic. Just like in conversation, some things need brevity in order to point out simplicity. On the other hand, there is Literate Programming, which I believe is for saner times than now.

[ Parent ]
And BTW... (3.00 / 2) (#18)
by slaytanic killer on Sat Dec 09, 2000 at 10:22:33 AM EST

The three Pattern Language books on architecture are fun to read, beautiful.

[ Parent ]
[Un] Dependable software | 21 comments (21 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!