Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Tying Income to Quality in Custom Software Development

By svillee in Culture
Sun Jan 06, 2002 at 12:36:29 PM EST
Tags: Software (all tags)
Software

An idea has been incubating in my mind for some time now, of starting a certain business quite unlike any that exists today, to my knowledge. What follows is a brief description of that business. Naturally I am interested in constructive criticism of it, but at the same time I have resigned myself to the fact that many people, perhaps most people, will think it is a bad idea. My main objective here is to get to know those who think it is a good idea. And if by chance someone in the world has already started such a business, I would love to hear about it.

In a nutshell, the business will be a custom software development company with a strong emphasis on quality. All development will be done by independent contractors, and the contracts will be designed so as to tie the developers' income as closely as possible to the quality of their work. They will not be paid by the hour. Instead, they will be paid based on what they accomplish.


At a broader level, the business will be a marketplace for clients who care about quality to find programmers who can deliver quality, and vice versa.

The People

The business will focus mainly on two groups of people:

  • Clients who are looking for specialized software, and have a clear idea of what they want. The ones I have in mind are skeptical of conventional wisdom about quality, e.g., that you get what you pay for, that the best products are the most popular ones, and that the best programmers are the ones with the most advanced degrees or the most years of experience.

  • Exceptional programmers. I mean the kind of people who can write thousands of lines of new code in a few weeks with hardly any bugs.

I am aware that these two groups are fairly small. It is a sad fact of life that many clients do not really know what they want, and of course it is tough to find top notch programmers. I don't care. I want to cater to these two groups anyway, even if the business does not make loads of money. The company will be owned and operated by programmers, and the goal will be to make a decent income with very high job satisfaction.

Also playing an important role will be people who excel at certain ancillary activities such as requirements analysis and testing. Their income will also be based as much as possible on the quality of their work.

Draconian Contracts

What will make this company unusual will be the terms of payment spelled out in the contracts. The terms will be quite harsh, arguably too harsh for many programmers. Each contract will specify an estimated delivery date along with the fee to be paid by the client if the software is delivered on that date. If the programmer delivers early, he earns an additional bonus linearly based on how much earlier. If he delivers late, a steep penalty is subtracted from the fee, again linearly based on how much later. Independent of this, there is a penalty for each bug found after delivery, scaled according to severity.

For example, the contract might specify that the fee is $50,000 if the software is delivered on March 31, with a bonus of $100 for each day early, and a penalty of $500 for each day late. And it might specify penalties of $25, $75 and $250 for each low, medium and high severity bug respectively.

A key provision of the contract will be that the programmer can suffer a net loss if he delivers late enough or if too many bugs are found. That is, if the penalties add up to more than the original fee, he will actually have to pay the client money. The contract will specify a limit to his liability, perhaps around $5000 in the above example.

The contract is considered over, and actual payment is made, after all reported bugs have been fixed and a specified amount of time (say one week) elapses with no new bugs being found, or when the programmer reaches his liability limit.

Escrow Agent

One of the major roles of the company will be that of escrow agent. Both buyer (client) and seller (programmer) will be required to deposit funds, typically in the amount of their maximum liability. These funds will earn interest for as long as they are deposited. When the contract is over, the accounts will be settled, and the parties will be able to withdraw any remaining funds.

Arbitration

The company will support an elaborate system of arbitration, designed to resolve disputes inexpensively but fairly. The arbitrator's decision will be enforced directly by the escrow agent, thus avoiding any need for a court of law.

The arbitrator will not simply make a decision. One of his duties will be to write an opinion, addressing each of the loser's arguments and explaining why they are invalid or irrelevant.

The parties may choose an arbitrator during contract negotiation to resolve any dispute that may arise, or they may specify that the escrow agent will choose an arbitrator in the event of a dispute.

The arbitration system will follow the "loser pays" model present in some European countries, meaning that the loser will have to pay not only the original debt, but also the arbitration fee and a certain compensation to the winner for time spent on arbitration. This is to discourage frivolous claims, and to encourage people to concede valid claims promptly. Also, interest on the original debt will be charged back to the date the claim was filed.

A key responsibility of the company will be qualifying arbitrators. A person applying to become an arbitrator will need to take a test where a series of hypothetical cases are presented and the applicant must choose the correct decision. In another test, the applicant will need to write opinions for hypothetical cases that are slightly gray.

The Requirements Specification

Each development contract will have an associated requirements specification. This document will be the fundamental reference for the arbitrator in the event of a dispute.

The specification must describe in detail the required behavior of the software, i.e., what it must do. A discrepancy between the required behavior and the actual behavior of the delivered software constitutes a bug.

The specification must not dictate the process by which the software will be developed -- this is always at the discretion of the programmer. In particular, the programmer is always free to subcontract any portion of the development, using whatever criteria he sees fit for choosing subcontractors.

At various times, the client and the programmer may modify the requirements specification by mutual agreement. In fact, this is likely to happen frequently. They may also agree to modify the estimated delivery date and the financial terms.

Bug Reports

Bugs may be reported either by the client directly, or by a third party acting as tester. The contract may specify that the delivered software will be made available to a certain group of testers, or even to the general public. Anyone so authorized may search for bugs either through actual testing or by reviewing the source code.

To be valid and eligible for a reward, a bug must cause a consistent failure. That is, the bug report must specify a "trigger scenario" under which the software consistently fails to meet its specification. It may often happen that the client observes an intermittent failure, but is unable to identify the trigger scenario that causes the failure to happen consistently. He should then give the testers as many clues as he can. Whoever finds and reports the trigger scenario first wins the reward.

In some cases, the correct description of the trigger scenario will require giving internal information, e.g., the sequence of task switches for a race condition, or the initial contents of dynamically allocated memory for a problem related to an uninitialized variable. Normally a specialist is needed for this, but the reward is open to anyone who can describe the trigger scenario correctly.

If two or more people report the same bug, only the first one gets the reward.

Requirements Analysis

In some cases, the client will write the requirements specification directly. But usually, the specification will be composed by a requirements analyst, someone who specializes in helping clients decide the details of what the software needs to do.

The company will not have a marketing department per se. The target client is skeptical, jaded and cynical, precisely the kind of person who would not respond to a conventional marketing campaign. Instead, one of the functions of marketing, namely locating prospective clients, will be performed by requirements analysts.

The most likely prospects, I think, will be upper management at companies with 100 to 200 employees. Companies smaller than this will probably not be able to afford our services. And in the larger companies, the decision makers are likely to be so enmeshed in bureaucracy and politics that it is difficult for them to care about quality. One characteristic of a good prospect is that he works closely with the people who will be using the software. In larger companies, software purchase decisions are often made by IT departments having little direct contact with users. Likewise, software vendors are not good candidates because they are even further removed from users.

A requirements analyst will frequently be a full time consultant who does requirements analysis when the opportunity arises. When in the normal course of consulting he encounters a client who needs something unusual, something not provided by off the shelf software, he may contact my company to negotiate with a programmer for a requirements analysis contract. First, though, he should do some qualification of the prospect. One good test is to explain briefly how the "draconian contracts" work, and see how the prospect reacts. If his eyes light up, that is a good sign.

A requirements analysis contract is similar to a development contract, except that the deliverable is a requirements specification instead of software. A kind of role reversal is happening here. The programmer, who will later be acting as seller in a development contract, is at this point acting as buyer.

The programmer may report bugs against the delivered specification. A bug in this context is an ambiguity, incompleteness, contradiction or inaccuracy such that the programmer cannot reasonably be expected to implement the software without further clarification from the client.

When all bugs in the specification are fixed and some time has elapsed with no new bugs being found, the contract is over and the requirements analyst may collect his payment. In particular, the payment is not hinged on the success of the later development contract, or even whether the development contract happens at all. This is intended to keep the requirements analyst focused on listening rather than selling.

Once the specification has been written, the programmer may enter into a commission sales contract with a salesman who will negotiate for a development contract. Again the programmer is acting as buyer, and here the deliverable is an executed development contract agreement, including deposit of necessary funds with the escrow agent. The salesman will be paid a percentage of the negotiated base fee. There is no concept of "bug" in this case.

Frequently the requirements analyst will be a good candidate to act as salesman, since he has already established a relationship with the prospective client. However, this is completely voluntary.

One hurdle the salesman will face is convincing the client that the escrow agent is trustworthy. As the company's reputation builds this should become easier, but it will never be a trivial step.

In some cases, the programmer will not have this burden of financing the requirements analysis. For example, when another programmer is subcontracting to him, the other programmer will be expected to write the requirements specification for the subordinate contract. Also, if a client will be posting a requirements specification for open bidding, then as a matter of etiquette he should pay for his own requirements analysis.

Company Services

In its role as escrow agent, the company will provide an Internet host that will offer services such as the following via encrypted messages:

  • An account holder may get status information about his account, and may transfer funds to his own bank when not blocked by an outstanding contract.

  • A client may post a requirements specification for open bidding, or for bidding by selected parties.

  • Either party may propose a contract, and the other party may accept it.

  • A programmer may report delivery against an active development contract.

  • A client may accept or dispute delivery.

  • A client or a tester may report a bug.

  • A programmer may concede or dispute a bug report.

  • An arbitrator may enter a decision and the associated opinion for a pending dispute where he is named as arbitrator.

A web site will also be maintained that gives general information about the company. It will provide services such as the following to anyone, even without an account:

  • A user may review the list of requirements specifications that have been posted for open bidding.

  • A user may review the list of active contracts that authorize testing by anyone, and may download the associated software.

The company will do little or no screening of programmers. Applicants will simply need to take a test of company rules, to make sure they understand the risk they are taking. However, new programmers will be restricted to contracts where their maximum liability is within some threshold, say $100. This restriction will be lifted when the programmer successfully completes a development contract with a net profit. This is my one concession to protecting people from themselves.

Examples of good requirements specifications and good software will be provided. In fact, the software that powers the escrow agency and the web site will be a flagship example of high quality software. It will be in the public domain (not GPL), and a substantial reward will be offered for bugs in it, say $1000 for each high severity bug.

The interest rate offered on deposited funds will be somewhat below what can be safely earned in the financial markets, and this will be one source of income for the company. A fee may also be charged for each contract -- this remains to be decided.

An employee of the company will be free to enter into a contract. In fact, this will be encouraged. The other party will be alerted that he is entering into a contract with an employee. The employee will need to deposit funds like anyone else.

Mally No Show

A client may deliberately enter into a development contract with a progammer where he fully expects the programmer to fail, that is, to reach his liability limit. The client is effectively betting against the progammer, the way the mob bet against Ben Shockley (Clint Eastwood) in The Gauntlet, but without the violence. This is not only allowed, but actually encouraged, as it helps to dispel myths about what programmers can do.

For example, a client may need to have a Java application written. He may enter into two development contracts simultaneously. One is with a "favorite" who has worked extensively with Java. The other is with a "Mally No Show", who has worked with C and Pascal, but not Java. The client expects to use the penalty from the second programmer's failure to help pay for the first programmer. Of course, if both programmers are successful, then the client must pay both of them.

Don' Worry 'Bout Your Funds, Mon

For a variety of reasons, I think it may be best to locate the escrow agency offshore. I will actually be on vacation in the Cayman Islands the week of the 14th, and I may do some research into this while I am there, even though the launch of the company is still quite a ways off.

Also, I think it may be good to allow parties to be anonymous. Of course, people can always refuse to enter into contracts with anonymous parties.

Conclusions

This is only a brief discussion of some of the issues involved with tying income to quality. I have not even mentioned certain topics such as maintenance contracts.

While I am reluctant to make a prediction, my goal is that the error rates (e.g., defects per thousand lines of code) for software developed in this manner will be at least two orders of magnitude lower than what is typical for conventional methods such as ISO 9000.

The company will have a direct effect only on a small community, i.e., the community consisting of the two groups mentioned at the outset. But as word of the company's success within this community spreads to the general population, some people may try to explain this success in the context of conventional wisdom. At that point, things could get interesting.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o Also by svillee


Display: Sort:
Tying Income to Quality in Custom Software Development | 46 comments (46 topical, editorial, 0 hidden)
Pay for performance (4.33 / 3) (#1)
by eyeflare on Sat Jan 05, 2002 at 08:57:50 PM EST

Pay for performance systems exist in markets where results are reliably quantifiable. Such as PR (# of inches in selected media for example) or telemarketing.

I believe your main problem will be balancing client spec changes and good measuring metrics so that everyone is happy. Apart from that, I'd buy this way.

"There is no way to peace; peace is the way." -A. J. Muste
That's nice, but... (4.40 / 5) (#2)
by fink on Sat Jan 05, 2002 at 09:07:43 PM EST

The first problem is defining Quality. That must happen in terms that all parties are happy with. That is not possible, as every individual, without exception, has a different view on what makes a Quality product. For example, what I consider an "acceptable" number of defects will differ from what you might consider "acceptable".

In particular, software developers will have a certain list of what makes a Quality product (low defect rates, speed, performance), and the clients of a piece of software will most likely have a similar, yet different, list (low visible defect rates, speed, ease of use). Oftentimes, requirements documents simply do not capture the actual Quality requirements (oftentimes, a "well-written" requirements document doesn't capture much at all!); try and define Quality without being qualitative and you'll soon see what I'm trying to get across.

While it's a very nice idea, and I wish you luck, I do see this raising more problems than it solves.

If I could find a certain WikiWikiWeb node which had a really good list of potential definitions of Quality, I'd post it here. I've read it before, but can't find it now. :-)


----

I agree with Fink: quality is VERY subjective ... (4.40 / 5) (#5)
by joegee on Sat Jan 05, 2002 at 10:30:17 PM EST

And unfortunately both you and your client must agree on the basic definition of quality for this to work. I can see it now:

Client: I don't like this piece of software.

Programmer: Does it crash? Did you lose data?

Client: I just don't like the way it looks. Its very harsh on the eyes, the icons look funny, and I can't see a big enough difference between the delete icon and the stop icon. It's confusing.

Programmer: Has it been crashing on you: is everything working OK?

Client: It's refreshing to not have that bug we had with the off-the-shelf product we used to use, let me tell you. Nope, no crashes, but the whole office staff are complaining about the delete icon. And Mrs. Smith's Yahoo chat quit working since she installed your software. You didn't test this with Yahoo chat, did you?

Programmer: We can change the delete and stop icons. I suppose we can make certain there are no conflicts with Yahoo chat.

Client: Am I going to have to pay for that too? And what about option 4 of menu 7?

It's a nice idea, but I suspect that any company that is paid based on the nebulous term "quality" is doomed to die the death of a thousand cuts. I know end consumers -- it's a love/hate thing. Your company's death will be slow, ugly, and very painful. :/

I love the idea of a company guaranteeing its software for quality. I deal with some ghastly ... -PEACHTREE- ... excuse me, a bit of a head cold ... anyways I deal with a bug-ridden product from a certain Company Who Shall Remain Nameless. The alternative is a custom-programmed accounting suite costing fifteen times as much as the current program. I would love them to guarantee quality. Every time one of my people complains about losing half a day's data due to a certain network error this company admits and refuses to fix -- I would love to have some recourse that didn't cost $90 per telephone call.

And no it's not my fault, I did not choose the software. How did this turn into a therapeutic rant? :) Anyhoo ...

*I* would pay extra for a quality guarantee, but you as the programmer and your customers have to be on the same page regarding your definition of the word quality, or disappointments and misuse of your guarantee are bound to occur.

<sig>I always learn something on K5, sometimes in spite of myself.</sig>
[ Parent ]
Reminds Me of Trilogy (4.50 / 4) (#3)
by Carnage4Life on Sat Jan 05, 2002 at 09:13:07 PM EST

Your company idea sounds a lot like Trilogy Software used to be and McKinsey supposedly still is.

Trilogy found out the down side to only getting paid when a project is successfully completed which is you may not get paid for discontinued projects and if you do it'll be lest than you budgetted for. Secondly, you don't get to bill by the hour for twiddling your thumbs.

Another problem with this model is that your programmers will tend to be expensive since they will be extremely talented while your competitors will have much cheaper employees (in fact some of the Big 5 tend not to higher technical people for technical positions but instead train non-technical people).

One would think that a company reputed to only hire the best that only gets paid when projects are successfully completed would be a hit but low demand for their services that would lead to multiple rounds of layoffs while a company that is known for overcharging, hiring non-technical people for technical positions and not finishing projects would be doing so well in a down market.

Anyway, good luck with your company and be careful of the Big Fish.

not really similar (4.00 / 1) (#10)
by svillee on Sun Jan 06, 2002 at 10:35:29 AM EST

Thanks for the pointers to Trilogy and McKinsey. I presume their contracts with clients are such that they don't get paid if they fail. But in most other respects, they don't seem all that similar to my company. In particular, they still look for the same old things in programmers: college degrees and years of experience.

...you may not get paid for discontinued projects...

That's why in my world the client has to deposit funds with the escrow agency.

...your programmers will tend to be expensive since they will be extemely talented...

I dispute this. My whole point is that talent often goes unrecognized. Look in any development group in any large company. There is one person that everyone within the group knows is a guru. They all turn to him for the tough problems. He writes high quality code, and spends a lot of his time fixing other people's problems. But his salary does not really reflect this. Raises are constrained by upper management to be no more than a certain percentage, say 10%. If the productivity ratio of the best to the worst is more than 10%, then obviously someone is not being paid according to his true worth.

And when his employer goes belly up because of the recession, he finds that in the outside world, nobody knows that he is a guru. He may have trouble finding a job because he doesn't know C# or whatever the latest fad is. Norman Matloff covers this topic at length in a wonderful paper.

My solution is to have a company where the only entry requirement for programmers is that they back up their claim of being an excellent programmer with a wager, i.e., by putting their money where their mouth is. You start small, with a liability limit of $100. The base fee is say $1000, and maybe you have a few bugs that take you down to $850 or so. You made a net profit, so your $100 liability restriction is lifted. Now you can take a bigger contract where the liability is $300 and the payoff is $3000 minus bugs and late penalties. If you're really good, you should make decent money after a while.

[ Parent ]

You've waaaay overcomplicated this (4.83 / 6) (#4)
by localroger on Sat Jan 05, 2002 at 09:19:39 PM EST

Well, at least at the smallish scale I operate. Our company has done this for years. Of course, we sell hardware as our main business; software is necessary added value to make the hardware attractive. (It's industrial.)

We interview you, analyze your requirements, and work up a flat-rate quote. If you accept it, we put the system in at an agreed date. If we don't have it ready, you can cancel the order and we eat any development work -- or you can ask us for a new delivery estimate, and tell us to proceed. (We had a lot of missed deadlines due to the y2k mess, but nobody cancelled.)

We put the system in. We plan on a few weeks' burn-in period while bugs and misunderstood requirements are dealt with. At a point around 1 to 2 months after installation, you either pay for it or we pick it up.

There's no byzantine penalty schedule. If we don't fulfill our promise, we don't get paid. Not a cent. And if you stiff us for any reason, we pick up our toys and leave you where you were before the sales call. No hard feelings. Call us when your funding is more secure.

In 15 years we have had to pick up the system exactly once. This occurred in the very early days when management did something I expressly told them probably wouldn't work, and it didn't. We have stopped work a couple of times when we felt customers couldn't or wouldn't pay, so we lost some development time but cut our losses.

I really don't see why it needs to be any more complicated than this. If it works you pay for it; and if it doesn't, we pick it up. Stuff that works really well is rare enough that most people will pay for it when they see how well it works even if they were planning to stiff you.

I can haz blog!

Yeah, right -- you're insane (3.42 / 7) (#6)
by J'raxis on Sat Jan 05, 2002 at 10:57:26 PM EST

It’s impossible to write a bug-free program. Yes, that’s what I said, completely impossible. Why? Because the libraries could be buggy, or the compiler itself, or the linker, or the underlying OS, or the filesystem the program resides on, or the hardware, and so forth. Unless you are planning on building a ground-up bug-free system first for your programmers to develop on, there will always be bugs. Even if these factors are as nearly bug-free as possible on the machine one of your programmers develops his code on, who knows what’s going on on the client machine?

At the very least, you’re going to need to make the client well aware of this; a lot of bugs a client could find during testing could be from his own shoddy OS or libraries.

— The Raxis

[ J’raxis·Com | Liberty in your lifetime ]

Scattershot thoughts (4.40 / 5) (#7)
by silent communication on Sun Jan 06, 2002 at 12:31:15 AM EST

Not only should this company be draconian, but it should also give every advantage to having developers succeed if they can make use of these advantages.

Main problems offhand you may wish to think about.

- Testing
Testers and testing software need to be reliable. If only the client has testers, life sucks.

- Difficult arbitrations
Arbitrations which are sufficiently difficult and reasonable should cost both parties. Sometimes, there are honest and reasonable disputes.

- Dev machines
This model assumes each programmer works with his/her own hardware? What if the programmer has insufficient hardware for the client's needs? If your dev company has testers, but the client's hardware is strange, isn't there some leeway if the product has unpredictable defects? Programmers rely on good testers, and good testers need to know their machines, and there is wide room for abuse during development time if the programmer must work in bad conditions.

- Escape clause
The programmer's quality is partly dependant on the client's ability to work together. A requirements document is simply not sufficient except in the most trivial of cases. How can the developer escape a contract without penalty if the client turns out to be impossible to work with?

- Objective arbitration.
They must be socially separated from developers, and their decisions should be made public within reason.

arbitration presents challenges (none / 0) (#15)
by svillee on Sun Jan 06, 2002 at 01:31:05 PM EST

I agree with most of what you say. My hunch is that you have given this matter some thought even before my article. I especially agree about the importance of making sure the arbitrator is not the client's golf buddy (or the programmer's).

Arbitrations which are sufficiently difficult and reasonable should cost both parties. Sometimes, there are honest and reasonable disputes.

Abstractly, I can see this might happen, and I really want my company to accommodate it. But off hand, I can't think of a concrete example of where strict "loser pays" is not at least tolerable. Do you have an example from personal experience?

The topic of development machines is tricky. Clearly, the model works best for PC development where the programmer probably has suitable hardware. But in real life the target platform may be such that the programmer needs to develop on a machine provided by the client. So the contract would need to protect the programmer from things like the client suddenly denying access to the development machine. It's complicated, but I have to believe the appropriate legalese can be worked out.

[ Parent ]

Challenges are spice of life :-) (none / 0) (#21)
by silent communication on Sun Jan 06, 2002 at 08:20:20 PM EST

But off hand, I can't think of a concrete example of where strict "loser pays" is not at least tolerable.
Maybe you could keep "loser pays" while letting the idea float around that the rules could be changed so an arbitrator might occasionally find it wise to charge both parties. If enough people come to believe this is a needed solution, a rule like this could be put into effect.

Your instincts may be right, and this could just be an abstract problem. So loser-pays will probably do fine.

The topic of development machines is tricky. [...] It's complicated, but I have to believe the appropriate legalese can be worked out.
I don't think these are deadly problems, just things worth anticipating. Anticipating situations breeds confidence.

Maybe it should be part of the culture, that a programmer responsibility is to voice their problems firmly, politely and early. The programmer has the best information because he/she's doing the work.

There should be some ease of communication among developers, because new programmers often don't know what is unreasonable working conditions. Either they're too afraid to complain, or complain wrongly. So there should be an informal... "community" or "watercooler" where people can chime in if advice is needed.

That way, in the worst situations, the project can be cut early. And more often, the two parties can confidently work out their problems, because it's usually not in anyone's favor to fail.

[ Parent ]

Interesting idea (4.50 / 4) (#8)
by TheophileEscargot on Sun Jan 06, 2002 at 04:51:29 AM EST

I think there could well be a market for a software house that specialises in "quality". However I think this idea is a bit too extreme to work in its current form. Some problems:

  • No marketing department? Uh look, we don't like marketers, but they do exist for a reason...
  • As we always hear, the big problem in software is maintenance. You can't tell the quality until it's been around for ten years or so.
  • Remember the 40/20/40 model? Or whatever it is now: the theory was that 40% of the project should be analysis/design, 20% coding, 40% testing/fixing. You're devoting a vast amount of effort to the coding, which should only be a small part of the project. Coding is trivial: design, analysis and v&v are hard.

    Even assuming the semi-mythical fact that a top programmer is vastly more productive than an average one, this generally applies to the coding only.

What I do like is the idea of a software company that won't do business without a detailed specification and sensible time/budget estimates. That would benefit the clients as well as the software house. If you've got those things, producing high-quality software is easy: you don't need to hire uber-expensive uber-contractors; just reasonably good people.
----
Support the nascent Mad Open Science movement... when we talk about "hundreds of eyeballs," we really mean it. Lagged2Death
I agree with most of what you say, but ... (4.00 / 2) (#11)
by Simon Kinahan on Sun Jan 06, 2002 at 10:36:10 AM EST

Even assuming the semi-mythical fact that a top programmer is vastly more productive than an average one, this generally applies to the coding only.

You've got this wrong. Its true - and much more believable - across the whole development cycle, from design to maintenance. I believe the original statistic is something like "lines of debugged code per day", and the range is something like 1-100.

Its not so mythical either. Its cited by Brooks in "The Mythical Man Month", and I believe its been confirmed several times independently, and with different metrics, that different programmers differ vastyly in productivity.

Simon

If you disagree, post, don't moderate
[ Parent ]

"Semi-mythical" (4.00 / 2) (#13)
by TheophileEscargot on Sun Jan 06, 2002 at 11:36:59 AM EST

I didn't say it was entirely mythical.

I've heard of a couple of studies, done over individual project lifetimes, that claim there are vast (factor 10) differences between the productivity of programmers, when the cost of bug-fixing has been taken into account.

This seems to have been the kernel for a slashdotland myth that really good programmers slam out code ten times faster than bad programmers. This can be counterproductive if it encourages speed at the cost of quality.

Regarding the studies themselves, a cynical bastard like me can't help thinking of circumstances where some programmers have been given easy and well-specified tasks, and other have been dumped in a morass of changes, poor specifications and near-impossible requirements. Were the productive guys really superprogrammers, or were they just the guys who got easy jobs while others were dumped in the shit? Without doing a very long-term study it's very hard to prove. I'm not aware of any such study. If anyone has a link to one, I'd love to take a look.

Regarding real life applications: even if superprogrammers exist, it's practically impossible to identify them from resumes and references.

Also remember that in software development, there is a strong sensitivity to initial conditions. The cost of fixing a bug over time rises very rapidly, so a small difference in the rate of bug-introduction can produce a large productivity difference. The actual difference between a superprogrammer and a normal programmer might be quite small. In real life you'd probably get even greater productivity benefits by introducing other quality measures, rather than go on a largely futile quest for superprogrammers.

Now:

...across the whole development cycle, from design to maintenance...

...the original statistic is something like "lines of debugged code per day...

So these superprogrammers are so good they're churning out vast amounts of debugged code in the design phase? I'm impressed! Maybe you should look up these studies and post a link.
----
Support the nascent Mad Open Science movement... when we talk about "hundreds of eyeballs," we really mean it. Lagged2Death
[ Parent ]

Myths and Otherwise (4.75 / 4) (#14)
by Simon Kinahan on Sun Jan 06, 2002 at 12:50:10 PM EST

There are definitely cross-project and non-project studies done showing similar differences in productivity. One good example can be found in "Peopleware" by DeMarco and Lister (and immensely worthwhile book anyway, incidentally), where developers competed in a coding war game, trying to outperform one another in time and defects on the same task. They found the usual spread: the best were 10 time better than the worst, and 2.5 times better than average, and the better half were on average twice as good as the worst half. They did include time spent on design (no analysis was necessary, because it was a preset problem) in calculating the time spent (which is what I meant above, incidentally - do I sound like a cretin ?). This seems necessary to me, as some people do more thinking up-front and less coding, whereas some do otherwise.

On your specific point concerning the nature of the work given out to people by project managers: those who study engineering productivity are usually wise to such things, having had many years experience, even though some project managers aren't. When it is measured - which it has been - the performance of teams follows a similar spread to that of individuals.

However, you are right in saying that these differences in productivity probably do not have the same sources as they are generally supposed to in slashdotland. In such circles, they are usually used to support the "hacker myth": that a bearded, asocial, unhygenic teenager living in his parents basement on pizza and jolt cola, working from 6pm till 9am every night can outperform Microsoft. Where it gets embedded in a groups's mentality this does immense damage: people work too long, stress themselves out, ignore their colleagues, and their productivity falls and becomes unpredictable. Where you examine the real work patterns of a "hacker hero", they don't look like this at all.

DeMarco and Lister make an excellent case that high performance correlates most closely with a good working environment: space, quiet, privacy, and control over interruptions. For people in the same environment, performance differed by 21 percent or less (much less that the 1000 percent for the community as a whole). Obviously there are other factors too, including "raw intelligence" whatever that is, the ability to maintain a state of flow regardless of environment, education, resources and so on. The things generally known not to be factors are: language, experience, and salary.


Simon

If you disagree, post, don't moderate
[ Parent ]
The actual reason (4.00 / 1) (#22)
by silent communication on Sun Jan 06, 2002 at 08:58:29 PM EST

Those lines/day of code are amortized through the entire project. In all likelyhood, the programmer is not banging out code faster (perhaps even slower), but is far more efficient and does not have such a terrible debugging phase.

[ Parent ]
I still have difficulty... (3.00 / 1) (#24)
by TheophileEscargot on Mon Jan 07, 2002 at 06:37:48 AM EST

...believing it happens across the whole cycle.

OK, coding is mostly thinking about problems, not typing. I can see super-programmers coding effectively 10 times faster. But I have difficulty visualising, say, a super-requirements-analyser, who interviewers clients 10 times faster, perhaps by speaking in a squeaky speeded-up voice. Or even a super-tester, who runs through the scripts 10 times faster, his fingers just a blur on the keyboard...

Yeah, I know it's the quality that is supposed to make much of the difference, but I'm still not convinced. There's enough subjectivity in studies (what counts as a defect? How much does a defect count against functionality?) that an academic can "prove" pretty much anything he likes.
----
Support the nascent Mad Open Science movement... when we talk about "hundreds of eyeballs," we really mean it. Lagged2Death
[ Parent ]

There you are (4.00 / 1) (#25)
by silent communication on Mon Jan 07, 2002 at 07:57:46 AM EST

Ah, but there are super testers, who can output more bugs than the rest combined, doing perverse things no one else would think of. And your requirements-analyser can be able to talk with you AND the client in a sophisticated way, leaving little to the imagination. In the same way, God save the organization that hires a sloppy sysadmin.

It's just that the whole programmer-as-God thing gets in the way. I think the best people aren't just faster or better in some metric, but have another dimension to draw on. Being 3D instead of 2D.

Unfortunately, many organizations treat these people like crap. Their loss.

[ Parent ]
Debugging (4.00 / 1) (#26)
by ucblockhead on Mon Jan 07, 2002 at 09:38:53 AM EST

But there surely are superdebuggers who can find bugs ten, twenty or more times faster than the average person. I'm convinced that the spread for debugging is even wider than for straight coding.
-----------------------
This is k5. We're all tools - duxup
[ Parent ]
The things is... (none / 0) (#27)
by TheophileEscargot on Mon Jan 07, 2002 at 09:56:23 AM EST

...a lot of this seems to fit the bill for what they call "junk science".

"Everyone knows" all these studies have been done. However, everyone's knowledge of these studies seems to be second- or third-hand: they read in a book that the study had been done, nobody has a nice set of hyperlinks to studies in peer-reviewed journals.

Some people talk about lines of debugged code, which implies it's to do with coding. Some talk about how it's the same for requirements analysis, design, coding, validation & verification: which seems like a heck of a coincidence to me.

I suspect that the vast productivity differences are true-ish, though probably exaggerated, for coding. But for the rest, it seems too much like the evidence for ESP: a few isolated studies, quietly ignored, except insofar as they fuel urban myths.
----
Support the nascent Mad Open Science movement... when we talk about "hundreds of eyeballs," we really mean it. Lagged2Death
[ Parent ]

Super-requirements-analyser (none / 0) (#38)
by mcherm on Wed Jan 09, 2002 at 09:00:30 AM EST

OK, coding is mostly thinking about problems, not typing. I can see super-programmers coding effectively 10 times faster. But I have difficulty visualising, say, a super-requirements-analyser, who interviewers clients 10 times faster, perhaps by speaking in a squeaky speeded-up voice. Or even a super-tester, who runs through the scripts 10 times faster, his fingers just a blur on the keyboard...
Gee... I can EASILY visualize the super-requirements-analyser. She doesn't speak in a squeaky, speeded-up voice, in fact she tends to ask FEWER questions. But they're the right ones. And then she writes up a document which is UNDERSTANDABLE, thus saving 3 cycles of code : present : back-to-the-drawing-board.

-- Michael Chermside
[ Parent ]
Yeah... (4.00 / 1) (#39)
by TheophileEscargot on Wed Jan 09, 2002 at 11:39:11 AM EST

...and by sheer coincidence the same ratios apply as with the super-tester, and the super-coder.

This is conclusively demonstrated by many studies which everyone's heard about, but nobody actually seems able to link to.

Say, have you heard about the car that runs on tap water? The oil companies covered it up, but... ;-)
----
Support the nascent Mad Open Science movement... when we talk about "hundreds of eyeballs," we really mean it. Lagged2Death
[ Parent ]

Links to studies (5.00 / 1) (#40)
by silent communication on Wed Jan 09, 2002 at 12:42:20 PM EST

I don't know if you're being sarcastic, but...

They're in books. I don't know where they are on the net.

Code Complete
Peoplewa re
Mythical Man-Month

[ Parent ]

Good idea, but some problems (4.33 / 3) (#9)
by cthugha on Sun Jan 06, 2002 at 05:51:11 AM EST

Firstly, as others have pointed out, quality is a subjective term, and is therefore a fertile source for arguments when it comes to payment. You therefore need a clearly-defined (as in "produces a number which cannot be disputed") quality metric (or metrics) against which the software product can be tested. You've suggested bug count as one such metric, which is good (although I prefer bug count per KLOC). However, there are other measures of quality, e.g. efficiency. You can specify a standard definition for quality, but my bet is that both programmers (who are notoriously pedantic about these things) and clients (who will have their own idea of quality based upon their needs) will complain long and loudly. Therefore, it's best to develop a quality metric for each job, included with the requirements analysis.

Since the client probably isn't up to defining a suitable metric for use in a software engineering context, they'll need to work with suitably trained testers (supplied by you) to formulated one. This already sounds like more than you imagined, as I feel you're thinking of implementing a simple clearinghouse or marketplace that wouldn't provide this kind of service.

On a side note, I notice that you only want exceptional programmers. Since the kind of people who would meet your requirements are pretty rare on the ground, I'm guessing you'd want to make this a global concern to build up anything approaching critical mass. That creates problems with regard to the enforcement of contracts, since each party is most probably in a different jurisdiction. Neither programmer nor client wants the hassle of going overseas to enforce their contractual rights; they'd probably be a lot happier if they dealt solely with you, since you rely on your reputation to act as a middle-man and therefore have a lot to lose if you don't deal fairly. Also, you should consult a lawyer who specializes in the area of trans-national agreements about the international law aspects.

The idea of an escrow agent is a bad idea. Getting 'exceptional' programmers to pay for the 'privilege' of working for you and your clients sounds like an exercise in futility. If they're that good, they're already employed, and have no need to take such a risk with their own money. If they're unemployed and desperate, then they probably won't have the cash anyway.

With regard to your contractual penalty clauses, I must say that the law tends to frown on out-and-out penalty clauses (i.e. clauses that are solely punitive in nature). The law will only enforce so-called liquidated damages clauses if they represent a genuine attempt to measure the damages suffered by the breach of contract they cover.

The idea of an arbiter is good, and is used as a form of ADR by many comapnies in almost any commercial situation where a dispute may arise, but like I say, you're going to have to think of alternatives methods of enforcement other than an escrow agent. I'm afraid the burden of enforcement will be on you, otherwise people just won't feel comfortable dealing with you. I know you're just trying to reduce your exposure, but so is everybody else.

I don't think the "Mally No Show" idea is very nice. I would simply let the client put his requirements out into the world, and let the market decide what is acceptable. There would be a need to create some kind of verifiable negotiating mechanism. Allowing two programmers to do the same job knowing that one will suffer is...unethical, and possibly illegal.

Oh, do locate your operation somewhere...reputable. Believe it or not, people don't feel very comfortable dealing with people in exotic countries with exotic (i.e. non-existent) finance regulations. Anonymity is a good idea, but this makes you fairly and squarely the midle-man. Legally, there will be no direct link between programmer and client, since neither will know each other, and you can't enter a contract with an anonymous person. When programmer and client enter into a contract, in reality, both will be entering into separate contracts with you. This puts you in a position of considerable responsibility.

In summary, to make this work, you'll need to make it more of a guild than a clearinghouse. You will have enforcement issues to deal with, and you should see a lawyer. Preferably three.



Interesting (4.75 / 4) (#12)
by Simon Kinahan on Sun Jan 06, 2002 at 11:10:30 AM EST

I like the idea of emphasising quality a lot. Indeed, its one reason I do my current job - although we are a conventional consultancy, not an agency type entity like you are describing.

I can see a few problems with your approach though. You seem to be going down a rather legalistic, contract-oriented path. In my view - and I believe this is a view largely endorsed by the legal profession - contracts only really work where they formalise already existing agreements and trust. The cases where total strangers can enter into an agreement without ever knowing one another's identities are very rare (derivatives trading is one, I guess), and very careful fomalisation is needed to make it possible. I don't believe we understand the process of software development well enough to make your scheme, or anything like it, work.

Given that I'm kind of reluctant to enumerate specific criticisms, but I will do so anyway, and try to indicate why extending your contract scheme to avoid these problems is unlikely to work:

1. Team based development. Most software is too big to be developed by a single person. Although there is a limit on useful team size (about 8 in my view), teams tend to be more productive as well as able to tackle bigger projects. Nothing in your scheme precludes teams making bids, I know, but it looks to me as if you are expecting the custommer to split the job up into single-developer work units at the analysis stage. That is unlikely to work well, because it precludes the formation of useful, gelled teams, and makes the problems of analysis worse.

2. Creating cast-iron specs is hard, or even impossible. Even the best analyst, with a cooperative client, cannot create a specification that perfectly describes what a system must do. Not only do clients lack the imaginative skills to forsee the consequences of everything they request, or to imagine everything they need, but vast quantities of usually expected behaviour are so obvious they are rarely specified. Clients rely on being able to report bugs that go outside the scope of the spec, and developers not only expect this but depend upon it. Developers expect some consideration from clients when they do things that are functionally equivalent to what is specified, but not exactly as requested.

3. Because of the above, trust between developer and client is absolutely essential. Selling consultancy, or development is a relationship business. The oppositional nature of your scheme - where developers lose out by admitting to bugs, clients lose out by accepting them as fixed - will damage that relationship.

4. Maintenance. Developing software - except for the relatively trivial jobs usually done by truly independent contractors (as opposed to temporary employees, or consultants) - is not a fire-and-forget business. Its rare to find people who are effective at maintaining other people's code. Clients know this. Contracts that end at the last bug are very rare in real life. Usually there are retainers, and agreements to at least quote on bugs found after user-acceptance. Payment for development may be made in tranches years after installation.

5. Developers submit the majority of bugs in most projects I have worked on. This is really important. One thing that helps support the relationship between client and developer is that developers often find bugs before the client, and fix them preemptively. The oppositional nature of your scheme will make this much less likely. Similarly, finding intermitent failures is often only possible with a knowledge of the code, and can be done much more effectively by the developer. Thats much less likely if they are going to be penalised for it.

6. The trust issues extend to your arbitration system. If the company is owned by the developers bidding for work, you'll end up with the arbitration process suffering from institutional capture: it will exist in the developer's world, getting its information from them, and will therefore be biased in their favour, whether it means to or not. Required judges to explain their decisions does not always help. The US supreme couurt recently demonstrated that some jurists are not averse to making a public mockery of the process.

Simon

If you disagree, post, don't moderate
developer not penalized for fixing his own bugs (3.00 / 1) (#17)
by svillee on Sun Jan 06, 2002 at 01:44:44 PM EST

Thanks for the comments.

On your point #5, I guess I wasn't clear about something. If the developer finds a bug, he is not penalized for it. (Or if you prefer, he is penalized for having it, and rewarded an equal amount for finding it, a net wash.) It does restart the one week timer, but I think that's appropriate.

Given that, I think most developers will be testing like crazy even after delivery, so that they will find the bugs, rather than someone else (which would cost them real money.)

[ Parent ]

Quality is job 1. (4.00 / 6) (#16)
by jabber on Sun Jan 06, 2002 at 01:37:44 PM EST

There is a big problem with the meaning of the word 'quality' here.

Quality can be pretty much defined as one of two things.. Technological correctness to spec, or customer satisfaction with the final product and process.

In most real world cases, true 'quality' is a combination of the two.

In the first case, dealing with correctness flirts with turning theory into practice.. The spec may ask for something seemingly simple, which carries many internal problems, making a quality product virtually impossible.

To avoid this, the spec must be truly complete, to the order of the specs used in the development of space shuttle control software, which are effectively, line-level pseudocode.

A customer who can provide such a spec has just written their own software, and would be better off getting a tool to translate the spec into a language of choice, than paying someone to do this manually.

When dealing with customer satisfaction, you end up with a product like Windows.. Put in enough 'cool' features, wave your hands a lot, throw in a free blowjob, and your customer will not care about a missed release date and a few bugs.

In short, it's a nice idea, but it won't fly. I'm sure there's plenty of Software Engineering professors who would love to tell their students that it will.

Take a cue from Total Quality people, and from Rational Software.. Expand the concept ad nauseum, into a 300+ page opus.. Send a draft to McGraw Hill.. Have it printed in book form and ride the 'silver bullet train' to fame and fortune. But know that in reality, it won't work.

What will is developing a process by which customers can develop unambiguaous, complete specifications.

But if you came up with a way to make programmers obsolete, you'll have the lot of us to contend with. :)

[TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"

the truth comes out (2.00 / 1) (#20)
by svillee on Sun Jan 06, 2002 at 03:13:24 PM EST

But if you came up with a way to make programmers obsolete, you'll have the lot of us to contend with. :)

Though it was evidently tongue in cheek, I have a feeling this is the unstated real reason many people will think the company is a bad idea.

Negative comments from prospective clients will carry somewhat greater weight with me than those from programmers.

[ Parent ]

Not quite (3.00 / 1) (#23)
by jabber on Mon Jan 07, 2002 at 12:09:51 AM EST

The proposal would put developers at a disadvantage, by binding their renumeration to something as subjective as 'quality'. In effect, it says that if the customer is 100% happy with the job the programmer did, they will be paid full fare, otherwise, less.

This will add huge arbitration overhead to every 'less than perfect' project, because a contractor who does not warrant full price will likely not be willing to accept responsibility for the shortcomings of the solution. We all know how hard it is to get a customer to fully define what they want, and to make sure that what they ask for is indeed what they want.

Even if you give people exactly what they ask for, if it is not what they want, they will not be happy, and the 'quality' of what you give them will thus be poor.

It's hard enough working in the current system as an employee, where each tenth of a percent of the annual raise must be argued for with management who make 'merit' decisions subjectively and often arbitrarily.

If you put the power to decide "fair pay" in the hands of the customer, where the relationship between the customer and contractor is already adversarial by nature, you will break the model.

Consider what would happen if each customer at a store was to set the price of each item based on their perceived 'quality' of that item.. And to pay for it after they use the item as they intend.. No, it isn't simple supply and demand - that works on large masses averaging out. If my livelihood depends on a fickle customer, I'll starve.

Granted, if a customer's payroll system is developed by a lousy contractor, the customer will go out of business as well. I see your motivation, but question the validity of your solution.

If I have a clearly defined contract to agree to, I can not do so with reasonable security, without already having the solution handy. All you're proposing to do is to shift the risk from customer to developer. That doesn't solve the problem at all.

The contract development process needs refinement, certainly. It needs to become more 'engineering' and less 'craft'. And you know what comes next... Professional Software Engineer licenses to guarantee 'quality'.

[TINK5C] |"Is K5 my kapusta intellectual teddy bear?"| "Yes"
[ Parent ]

capitalism (3.00 / 4) (#18)
by kubalaa on Sun Jan 06, 2002 at 02:49:51 PM EST

I took one glance at the title and thought, "Isn't that basically the point of capitalism, and doesn't it happen automatically?" That's an oversimplification, of course, but no matter what rules you come up with inside your business, "quality" and "income" are both going to be set by the marketplace.

Game theory (4.16 / 6) (#19)
by ucblockhead on Sun Jan 06, 2002 at 02:55:24 PM EST

There's a couple problems you are going to have. The biggest is that in the scenario you've set up, there is a huge disencentive for the client to create a good specification. The worse the specification, the more wriggle room for changes, the better off the client is.

Now you can say "but the programmer won't accept a bad specfication", but that's niave as very few have the sort of legalistic minds necessary to ensure that there are no loopholes in the specfication. Programmers are notorious for underestimating. This is true at all levels of ability, even expert, guru programmers.

All this means that the client is best off giving a very aggressive date, before they really need the thing, and to specify as little as possible. Getting the specification is going to be a huge battle. These things are already true in normal situations. Your scheme will encourage it.

The second trouble will be in finding good people. You are asking people not only to take huge risks. You are also asking people to defer any salary for the entire project. Not many people can do this for a six month project. You are going to be competing for people with companies paying high, stable salaries with benefits.

I can tell you that I wouldn't be caught dead in such a situation, not because I think I'm a bad programmer (in fact, I can point to a long chain of on-time projects with few bugs) but because I've got a mortgage and a wife and I simply can't afford the risk.

And finally, while the differences in productivity between good and bad programmers are well known (as high as 25 to 1 in one study) it is a truism that everybody things they are the twenty-five and not the one. You'll get a lot of crappy programmers who talk well and sound good, and they'll fail on a project, and while the programmer takes the money hit, the client will see it as a company failure, and that will make getting more clients difficult even if the programmer to blame leaves.
-----------------------
This is k5. We're all tools - duxup

I see you're a fan of quality (4.33 / 3) (#28)
by epepke on Mon Jan 07, 2002 at 03:09:09 PM EST

I'm sure you know that there are several similar escrow agencies available online. I've gotten some money off of them, too.

Anyway, I see you're a fan of quality. So am I. Good, solid, bug-free code that does the right thing and doesn't break with the new release of an OS. The trouble is that quality is already tied to income--negatively. To put it simply, bad sofware makes more money.

This is most obvious in the auxiliary revenue streams, including

  • Upgrades
  • Help incidents
  • Consulting
  • Books

Less obvious but also important is the fact that a bad piece of software doesn't sell as well as a good one. Good software usability is positively correlated to how unobtrusive it is. Software sales are based on how impressive it is. These goals are diametrically opposed. Of course, you can get bad software to market faster and spend less on writing it, but that's gravy.

Of course, software that is too bad may not sell at all. However, people have been so trained in low expectations that it has to be really, really bad in order to get hit by that.


The truth may be out there, but lies are inside your head.--Terry Pratchett


I agree (3.50 / 2) (#31)
by svillee on Mon Jan 07, 2002 at 07:12:53 PM EST

You are quite right about the disincentive to pay attention to quality.

I personally find it apalling (but not surprising) that Microsoft charges something like $290 for a customer to make an official bug report. That's quite a racket they have going.

I'm sure you know that there are several similar escrow agencies available online. I've gotten some money off of them, too.

Could you give me some references to these? I would really appreciate it.

[ Parent ]

Refs (4.50 / 4) (#33)
by epepke on Tue Jan 08, 2002 at 10:14:48 AM EST

The one I'm most familiar with and have the best experience with is hotdispatch.com. There's also elance.com and a couple of others, but I can't remember their names, and I wasn't too impressed with their methods of negotiating for payment.


The truth may be out there, but lies are inside your head.--Terry Pratchett


[ Parent ]
US Gov already does this (4.00 / 2) (#29)
by karb on Mon Jan 07, 2002 at 03:32:11 PM EST

For some (but not all) contracts you have something called an "award fee." It varies, and I'm no expert in government contracts, but the government will award a percentage of a predefined sum to a contractor based on the contractor's performance. It's quite a cool system, actually. It makes you pay a lot of attention to the customer.

btw, I'm not sure how they ensure that it's impartial, but it appears to be.
--
Who is the geek who would risk his neck for his brother geek?

roadwork (5.00 / 1) (#30)
by ucblockhead on Mon Jan 07, 2002 at 07:05:07 PM EST

This is similar to how the contracts that rebuilt the LA freeways after their last earthquake were written. It worked very, very well. But there are a couple key differences. These were companies, not employees signing the contracts, and corporations can take financial risk far more easily. More importantly, freeways are fairly easy to specify, so argument about whether or not the date was met were minimal.
-----------------------
This is k5. We're all tools - duxup
[ Parent ]
Critical flaw. (5.00 / 5) (#32)
by jayfoo2 on Mon Jan 07, 2002 at 10:14:25 PM EST

You specify that you will only hire and use exceptional programmers. That's going to be an issue. An exceptional programmer can earn just about as much as they want anyway. Without the risk associated with your approach.

Breaking it down.

To make this a worthwhile proposition for your employees you will need to start with a much higher rate/price than you would with a traditional consulting contract (which my company is very good at btw, and we've got a better cost structure that you're going to, we're big).

If you don't start with a higher price then you (and your employees) will get killed. Put simply you will never collect 100% of your initial price. Won't happen. No one is perfect. No client will never disagree. You'll get stuck negotiating.

So you either land up earning less than my company would (because you started at the same place) or earning the same (if you started higher).

You'll get killed because no client is going to go with your higher price to start with (adjusted for perception of quality, but mine's good too). When clients do software projects they start with a budget that usually is comprehensive (actually it's usually low). They usually can't go to an executive and say they're intending to spend more, unless they are planning to screw you. So if you want to do any work you'll have to be price competitive with me. Then you'll get paid less when all is said and done.

Long story short after one or two projects your employees will come work for me (or my ilk). Why would they face the risk of your scheme.

Yes this analysis has two assumptions.

1. Clients will not budget a premium for you
2. Your employees can't execute well enough to beat me

I base those on 5+ years of consulting management and sales. For the first one trust me that I can kill you by going in at a lower price. My company looks (and is) very impressive. I've got plenty of referenceable clients who will tell any prospect that not only do I rock but I also roll. You'll have a very hard time providing the client a big enough difference in quality perception to command the premium you need (once again unless the client is planning to screw you).

The second assumption is once again based on my experience. I've never been involved in a project that didn't have bugs, especially if you are going to let the client find them for a long time. Trust me that my programmers are just about as good as any you will find. My quality is going to be pretty close to yours, once again without the risk.

The final thought is that you just don't need to do this. Clients are happy to engage consultants with more favorable terms. Why would you ever sign a worse contract than you have to. You can always do a good job (because that is the moral thing to do), but why risk yourself unneccesarily.





Guess I have to write some more (5.00 / 7) (#34)
by epepke on Tue Jan 08, 2002 at 03:03:45 PM EST

Avast there matey, there be rants ahead.

I guess my other response, "I see you're a fan of quality," was insufficient, even though you agreed with that. So here's more.

I'm 40 years old. I started developing real software for people at 16; it was a payroll system for a string of hotels. I've been doing software development ever since, so I remember things that only a vanishingly small percentage of people are even aware of.

I am also one of those pedantic bastards, to borrow a phrase from some other article, who is very concerned about making things work properly. When I went to University, it was popular to prove every algorithm--that's the easy part. So I made my specialty graphics and user-interface design--that's the hard part.

Over the past 15 years or so, I have seen software get worse and worse. With a couple of notable exceptions, such as game and industrial control software, the quality of software is significantly worse than it was 10 years ago. Oh, software is bigger now. It uses more disk space and generates more temp files. It uses more hardware (which unlike software, has improved tremendously). Sometimes it does more, though this is a double-edged sword, as the disease of featuritis demonstrates. Don't get me wrong--I don't really want to go back to using WordStar on an ADM-3A; I like WYSIWYG, although there are still some purposes for which TeX is a better choice. But it never broke, was customizable, and was more responsive on a 1MHz machine than Word is on a 500 MHz machine. And it was written in assembly language, with overlays, and "everyone knows" that makes programs flaky. Software doesn't get like that without people who pay attention to quality. It just doesn't happen any more, and it isn't because everyone who knew that kind of craftsmanship died (although they do find it difficult to get work).

So, for the past couple of decades, I've been paying attention to new developments in software. Every new language, system, and advance in software "engineering" which is supposed to solve all our problems but never quite does, advocated by people who get really ticked off at being asked to show some evidence that it works. Every user interface book by people like Donald Norman and later Alan Cooper, even as they went from inspiring and helpful to acrimonious and nasty. The acrimony is directed in a way that is similar to what you are proposing: to the developers. What would fix it always changes: internet appliances, design professionals, CASE tools, XML, whatever. At first I believed this and that the solution was better to educate developers. In February, 2001, I finally realized that I was wrong and had been wrong for years. It threw me into a severe depression for two weeks.

The precipitating event was this. I was reading an Alan Cooper book called The Inmates are Running Asylum. Mostly a rant about software developers but highly recommended by Those In the Know, it contains a piquant piece about how he so much wants to have a VCR with a dial to set time and about how he goes into Circuit City waving a thousand dollars asking for one. I checked the copyright date, and, about the time he was writing the book, I remembered that a universal remote that let you record shows using a clock face that you turned. Perfect. However, he obviously didn't buy one, or he would have written about it in the book! I didn't buy one either; my excuse was that I didn't have any money at the time. Now I have money, so I decided to get one. I searched all the local stores, the web, and 800 numbers, and nobody had one. Only one person I talked to had even a vague memory that such a thing had ever existed. "It was cheap, too," he added. One VCR store owner said "We don't stock esoterica." It is as if in the space of three years the very idea of such a device has vanished off the face of the Earth. The problem isn't that it wasn't developed. It was developed and sold and marketed by someone who surely went out of business and lost money, but people, up to and including Alan Cooper, who couldn't be bothered to get off his butt or reach for a telephone to investigate direct evidence against his conclusions, refused to buy it. In droves.

Developers aren't the problem. The problem is that they are required to write bad code, partly by their managers, but mostly by the marketplace.

Every piece of commercial piece of software that you've ever seen break, screw up files, frustrate the user or not do what it's advertised to do, is something that somebody bought, and they bought it because that's what they wanted. Sure, they may have gotten frustrated at first, but eventually they overcame their frustration. Here's the kicker. All other things being equal, and within some bounds, the greater the initial frustration, the greater their esteem of the product. Once they've learned the software, they have some pride in the fact and may gain social status. A bug isn't a bad thing to them; it may frustrate them, but it has two bonuses: the satisfaction when a workaround is found, but mostly their ability to feel superior to the machine, which is still and probably always will be the steam machine that John Henry fought, unless it's worse. I know that this doesn't affect thee and me, but the computer is an object of fear for most people: just look at 2001, Alien, Colossus, the original Star Trek: "The Ultimate Computer," Demon Seed, or just about any cultural depiction of a computer that isn't a saccharine-cute robot. Sure, people have computers in their homes nowadays, but they still have a low-level fear of them, which they often project into demonization of hackers but which also is a significant influence on their entire purchasing and use experiences. They need something to counter the fear, and it's called "image." A few artists and musicians and people in advertising are going to go for the Northern California Zinfandel-and-hot-tub image that Apple caters to, but most are going to go for the tough, no nonsense, no-pain-no-gain Puritan work ethic image, which they call "professional" or "businesslike."

Anyway, you get experts, and they spread the word. People know that they are the ones to ask. "Hey, Joe, what to you think of Widget Mangler?" "Well, Bob, it takes some getting used to, but once you do that, you realize that it's a Real Professional Widget Handling Application. Oh, it crashes a few times, but what do you expect? All Software Has Bugs." "What about Widget Fondler?" "Well, Bob, huh huh, you know, Widget Fondler may be OK for your grandmother, but it's not a Real Professional Widget Handling Application." None of this has anything to do with what the packages can do; Joe would say this even if Widget Mangler had a hard-coded 2K limit; dealing with that limit would just make the "professionalism" all that much sweeter.

Don't believe me? Think I'm a grump with an attitude problem? Try it. Post any criticism of any software package anywhere and count the people who say, "I've never had that problem" or "I'm an MCSE, PhD, MOUSE, and that should never happen on a Properly Configured System which I (be impressed now!) Know How to Do, you luser." These are the people that others listen to when asking around about software. Unless they are tremendously outnumbered, they win.

Want to know how strong this is? A few years back, I kept track of which brands of condoms broke more or less often and posted information thinking that in the days of AIDS people might become a teensy bit interested in the quality of the condoms they buy. Invariably, someone wrote a response something like, "Well, I've been using condoms for 20 years, and I've never had one break." See? People like to do this so much they'll do it even if it means publically saying "I have a tiny penis and premature ejaculation."

Whenever I say something like this, people get ticked off at me and say things like, "well, maybe you don't understand what quality really is, you dork." That's pretty much my point. I don't understand how XP, which was released with a big security hole that Microsoft claimed was the first direct remote access security hole in a Windows product (which I don't believe, but that's unimportant) can simultanously be the most secure Windows system ever. Probably an idea by the same folks who came up with "Quality Assurance." I'm not sure whether the possibility that they don't know what "assurance" means in English or the possibility that they do is more frightening.

Or they say something like "software is more complex now" or come up with some other ad hoc justification. Well, first of all, most of those complexities are things that were supposed to make software better and easier to develop: API's, objects, etc. Second of all, I don't buy it. Show me a "complex" word processor, however hairy, and I'll show you a scientific package with a million lines of code, in FORTRAN 66, that uses computed GOTO's, and that always works. Or even the Space Shuttle code, which is also solid, and was developed under quality, complexity, and reliability constraints that would make you weep to hear them.

Your escrow service, great idea. I love it. I'd love to join. It won't work. Nice idea; shame about the reality. Unless you can come up with a way of removing the disincentives to producing quality software and doing so on a large scale. Either the software you produce won't be better than average, or it will be. In the latter case, you'll lose to Joe's Really Good Professional Software down the street, where Quality is a Slogan.

Come up with a real solution, and I'll be impressed. Otherwise, I'll continue to hide in the garret of in-house software development, where I'm the only game in town, and work on the side with Open Source, where I won't be penalized for writing software that isn't "commercial quality."


The truth may be out there, but lies are inside your head.--Terry Pratchett


caveat orator veritatis (5.00 / 2) (#35)
by svillee on Tue Jan 08, 2002 at 08:50:22 PM EST

Sometime around the year 1514, Nicolaus Copernicus conceived of the idea that the Earth revolved around the Sun. But he feared criticism from religious leaders such as Martin Luther, and it wasn't until nearly 30 years later that his book De revolutionibus orbium coelestium was finally published. Ironically, Copernicus died the same year. In 1616, Cardinal Robert Bellarmine decreed that Copernicanism was "false and erroneous", and on June 21, 1633, Galileo Galilei was found guilty of having "held and taught" Copernicanism, for which he was sentenced to house arrest for the remainder of his life.

Charles Darwin travelled with the HMS Beagle from 1831 to 1836. Another 23 years elapsed before his book Origin of Species was finally published. Again, the delay was primarily due to fear of criticism both from comtemporary biologists such as Richard Owen and from religious leaders. Even today, the debate rages on as to whether the theory of evolution should be taught in schools.

The ideas of Copernicus and Darwin were received with hostility not just because they disagreed with contemporary wisdom, but because they challenged our vanity. We thought we lived at the center of the universe. Copernicus said no, the earth is just another planet orbiting the sun, and it later turned out the sun is just another star in just another galaxy. We thought the first humans were created on the sixth day in the image of a deity. Darwin said no, humans evolved naturally from other animals over millions of years, just as those animals evolved from still others.

The human race needs to get over itself. That was a common theme of what Copernicus and Darwin were saying. The scientific theories of these men have since gained fairly broad acceptance. Yet it seems, if anything, we have become even more vain.

I salute you, sir, for saying what people don't want to hear. I salute you for having an original thought, one that has not been deposited in your brain by the charismatocracy.

If I had an unpopular opnion, I confess I would be inclined to keep it to myself. I would probably strive to arrange for visible events to happen that put my opinion to the test. In this way, I might occasionally spotlight the uncomfortable truth without actually speaking it.

You say that my company will not work. If by "work", you mean make lots of money, then I agree. If by "work", you mean make the masses understand that there are enormous differences in productivity and quality from one programmer to the next, then again you are quite correct. That is not likely to happen any time soon.

My goal is much more modest. I want to get to know members of that small community of people who care about quality. Such people will tend to find the "draconian contracts" appealing.

Hi. My name is Stephen Villee, and it is truly a pleasure to make your acquaintance.

[ Parent ]

Copernicus and Galileo (none / 0) (#42)
by HoserHead on Wed Jan 09, 2002 at 02:00:52 PM EST

This is off-topic, but a subject which is somewhat important to me.

First, it's not an irony that Copernicus died the same year his book was published. He refused to publish it because he was a very timid man, and afraid of being ridiculed. It's thought that the published work was read or shown to him on his deathbed, but I don't know whether that's true.

Second, in the case of Galileo and Copernicus' findings: They were penalised by the Church not because it went against religious teachings, but because the Catholic Church was embroiled in a political battle with the Protestant, and admitting they were wrong on this count would be a serious admission of weakness.

The fact is, nobody in the know said that Galileo and Copernicus were wrong. In fact, the Church's astronomers agreed with them. The church just said "Listen, you're right. We know you're right. Just shut up, will you?"

This kind of myth, that the Church refused to acknowledge their works based on the Bible, is one of my least-favourite, yet popularly believed myths. My very least favourite, of course, is the myth that people before Chris Columbus thought the earth was flat. Everybody knew it was round - hell, the Greeks (who, if you'll recall, were around somewhat before Mr. Columbus) even measured it, and accurately!

[ Parent ]

WAY off topic... (none / 0) (#44)
by ucblockhead on Thu Jan 10, 2002 at 10:55:41 AM EST

It's even better than that. The Greek measurement was not only relatively accurate, but also well known at Columbus' time. Given that it implied something like 12,000 miles of open ocean between Europe and China, you'd have had to be completely crackers to try an around the world voyage. Columbus was working on "new" calculations that were horribly inaccurate but made the journey look doable. That was one of the reasons he has so much trouble getting backing. All the experts says he was crazy, not because the earth was flat, but because they thought he was underestimating the size of the earth. They were completely correct.
-----------------------
This is k5. We're all tools - duxup
[ Parent ]
The reason (4.50 / 2) (#37)
by silent communication on Tue Jan 08, 2002 at 11:52:52 PM EST

Get this man his own damn website.

I have been thinking of this often. But if you look at it, the reason is because of the unholy intersection of programming and business.

Why are software projects so "hard" to manage? Because it is programming, with the additional constraint of a business case. XML and Java are so hyped because programmers are lusers too, and they're a demographic that buys products they don't really need, working to spread platforms they don't own.

The reason things don't work is this: Business. No one wants to design the 10,000th VCR. It's been done. Most programmers don't even want to be programmers, except for the promises of money.

Businesses exist as devices to generate money. They're a meta-program, a philosophy, whose purpose is to part people with cash. Symbolism is one way; by drinking Coke, we are part of a movement. The sexy model in the pop-up ad statistically increases the likelihood of people pointing with a mouse cursor and touching her. Holy wars between text editors erupt because we are afraid that we are tiny without our tools.

Quality is one thing. Making money is another. The only reason there is a correlation between the two is that the APPEARANCE of quality is one strategy employed in making money.

That is the reason why this new "quality" company will make only modest amounts of money, sort of like Ben & Jerry. It is a business, with the constraint of having actual quality.

[ Parent ]
Admirable goal (4.00 / 1) (#36)
by calimehtar on Tue Jan 08, 2002 at 10:15:34 PM EST

I think you underestimate the the number of clients with deep pockets who are interested in quality, as well as the human qualities of the parties involved. My experience has been that the best way to encourage quality among your programmers is to give them a sense of community, of responsibility and of trust in you. Be honest with them, pay them well, and expect nothing but the best from them. They'll reward you with dedication. Instill a sense of respect in your clients by setting realistic goals and meeting them. Charge a lot of money for your services and do a good job and you'll find yourself in the position you're looking for, and without all those contracts and escrow services. It's a proven fact -- people think expensive things are better whether they actually are or not.

+++

The whole point of the Doomsday Machine is lost if you keep it a secret.


You'll be just another consulting firm, to me (none / 0) (#41)
by imrdkl on Wed Jan 09, 2002 at 01:04:14 PM EST

Customized solutions delivered on time are what you all promise, but rarely deliver. At least, not unless you are familiar with the value chain and existing (legacy) systems which exist within most potential large clients. This knowledge of how a company really works inside takes years to obtain. In fact the ability for a developer or analyst to maneuver between and among these systems (and egos which control them), is the missing link that so many of us regular employees and managers which utilize the services of consulting firms are looking for.

And the search continues.

I submit, in fact, that such a consultancy is little more than a pipe dream, until and unless a brain-dumping tool is invented. To demonstrate, your target audience of upper management for requirements analysis, while logical in a business sense, will rarely, if ever, allow you to hit the target for delivering something which actually works better than the legacy system(s) which you are trying to replace.

I confess, I did not scrutinize your entire plan. Perhaps you have thought of something that others have not, or perhaps your stringency will somehow attract and encourage pragmatism on both sides. But I really dont see how you are gonna make any serious money in this venture, unless you adopt a more pragmatic approach, meaning that do not work for timely deliveries, but rather to extend your contract through at least 3 major revisions, at which point so much money and time will have been invested by those of us who are trying to tell you what to create that we wont have any choice but to allow you to continue.

I dont mean to sound cynical here, but that's how consultancy firms make the real money.

legacy systems (none / 0) (#43)
by svillee on Wed Jan 09, 2002 at 08:06:33 PM EST

Thanks for the comments. Most of the earlier replies have been from programmers, and it is good to hear a client's point of view.

...your target audience of upper management for requirements analysis...

I regret I was not clear about this. I meant to say that the most likely prospects for clients (in the sense of decision makers) will be upper management at companies with 100 to 200 employees. But the requirements analyst would not only talk to the decision maker. In fact, most of his time would be spent talking to the users of the legacy system. These people in the trenches are the ones best able to suggest how the system can be improved.

One of the fundamental jobs of the requirements analyst is to document the behavior of the legacy system, or at least those functions being retained in the new system. Sometimes, you get lucky and there is good documentation already available for the legacy system, which can be copied and made part of the requirements specification. But usually the documentation is poor or nonexistent, and then things get challenging.

...to extend your contract through at least 3 major revisions...

I may be a naive businessman, but even I can see this is a good plan. The contract is officially over once the week goes by with no bugs found, but of course later we'll be calling you to see if you need any revisions or enhancements. The best clients are repeat clients.

I gather you have had a few experiences with consultants not delivering what they promised. Are there specific cases where you feel you would have difficulty convincing an arbitrator that it was a bug?

[ Parent ]

The problem is not bugs (none / 0) (#46)
by imrdkl on Fri Jan 11, 2002 at 12:27:36 PM EST

Typically, bugs are the least of our worries, unless they affect the data-quality. The big thing I've seen is an UI that is supposed to look and feel the same as the legacy app, but just doesn't measure up.

Get the new/replacement UI working right, and the speed, and even data quality can suffer for awhile. But when it comes to redesigning an UI, I think you're gonna get in trouble with your current guarantees in your contract.

Good luck, anyways.

[ Parent ]

Great goal (none / 0) (#45)
by epepke on Thu Jan 10, 2002 at 04:02:19 PM EST

My goal is much more modest. I want to get to know members of that small community of people who care about quality. Such people will tend to find the "draconian contracts" appealing.

Great goal. However, even if you don't need or want to make a lot of money, you do need to survive. You need to get customers, otherwise there is nothing to do. You need to avoid getting smooshed, otherwise you're just a pretty red stain on the sidewalk. Fifteen years ago, it would have worked. Not today. Just like was for all intents and purposes impossible to start an effective commune in the late 1980's, this is effectively impossible today.

You will find people interested in quality. You will find them in Open Source projects. You will find them in online programming competitions. You will find them in universities. You will find a lot of bozos, too, but that's life.

This is why I put such hope in Open Source: because it has mostly broken free from the constraints of business revenue generation. But all is not great. Notice that people have started criticizing Linux in the past few months, and notice how this followed multiple corporate distribution vendors? I think that conditions are so bad now that it for all intents and purposes impossible to play nice within that stream; some sort of schizm is necessary. My schizm is this: instead of being a research scientist, which I used to do and was more fun, I whore my skills for a lot of money. Then I smile at the big pile of bills, buy stuff I need, and subsidize my own desire to build stuff that's good. It's too bad that it's necessary, but it's better to acknowledge and accomodate necessity rather than ignore it and become corrupted.

Hi. My name is Stephen Villee, and it is truly a pleasure to make your acquaintance.

Nice to meet you, too. My name is Eric Pepke, and most people know I'm a cynical git.


The truth may be out there, but lies are inside your head.--Terry Pratchett


Tying Income to Quality in Custom Software Development | 46 comments (46 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!