Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
How does a Open Source Software Project get started?

By billnapier in Technology
Mon Oct 09, 2000 at 02:12:45 PM EST
Tags: Software (all tags)
Software

Since the dawn of time, hackers have been driven by the need to solve problems. I have a problem, and the inclination to solve it, and don't think that anyone else has solved it before (at least that I could find). Since I think other people may enjoy my solution, I intend to open the source to it and let other people use and improve it. But at what point during the project is the right time to open the source?


It is my understanding that a majority of projects that end up being open-sourced, start out as a small team of hackers cranking out code. Once the code has reached a "stable" state, the doors are opened for people to use and improve. But is this the best approach?

What if once a problem has been identified, the lead hacker (the one who had the idea to begin with) starts soliciting help (like setting up a project page on Sourceforge) of other interested hackers? You would be opening up the source code even before the code has been written.

While an interesting idea, it may have some problems. With so many people working on a new idea, there is the possibility that it will never get finished, or will only slighty resemble what the original intention was.

So does the adage "Too many cooks spoil the stew" apply to open-source projects in the beginning phase?

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Poll
When should a problem be open-sourced?
o As soon as the idea occurs. 12%
o After the initial design. 12%
o After the first implementation. 53%
o After a beta-release 17%
o Once my company is losing revenue on the product. 3%
o Never. 1%

Votes: 64
Results | Other Polls

Related Links
o Sourceforg e
o Also by billnapier


Display: Sort:
How does a Open Source Software Project get started? | 11 comments (11 topical, editorial, 0 hidden)
Wait until you have something before you open (3.84 / 13) (#1)
by renec on Mon Oct 09, 2000 at 12:07:33 PM EST

I recall seeing a large influx of 'crap' project-postings to freshmeat shortly after open-sourcing became popular enough for the internet drones to notice it. Suddenly everyone with a project idea who could write 'hello world' in visual basic was announcing a codeless open-source project and soliciting programming help. Of course, they wanted to be team-leads and have creative control on the project, as well as getting all of the glory.

Suffice to say, almost none of those projects went anywhere, as far as I know. Nobody wants to 'contribute' to a programming project that has very little->no code written for it. Open-sourcing a project shouldn't be about getting programmers to work for you for free.

If you don't have enough working code to be at a point where you can justify distributing a binary of some sort, providing the source for the project isn't much use.

it already exists (3.40 / 10) (#2)
by mikpos on Mon Oct 09, 2000 at 12:13:37 PM EST

You're describing what's possibly the biggest problem with open source software right now. People come up with an idea, they make a pretty web page, set up mailing lists, CVS systems, hold logo competitions, etc. Then, a year later, the project is undoubtedly dead. If you're lucky, there has been a single line of code written.

Personally, coming into a project a few months (or years) after it's started is no big deal. Every project gets a complete rewrite at least once. Enlightenment (the window manager) seemed to be rewritten from scratch every other day. I'm not saying that that's a good thing (it's obviously a result of poor planning), but it means that even if you don't get into a project at a start, it does not mean that you will forever be condemned to live with the fundamentals designed by someone else. Just wait for the next rewrite and you'll be able to change the fundamentals.

You need a first implementation before going open (4.09 / 11) (#3)
by Mendax Veritas on Mon Oct 09, 2000 at 12:38:48 PM EST

You really need to have a small, closely-knit team to do the first implementation and establish the architecture. Open-source projects need to have a good, working framework in place before random people from all over the world start adding to it, or else it will quickly turn into chaos (assuming, of course, that the project attracts any interest).

The established pattern is something like this: Joe Hacker gets an idea, maybe talks it over with friends, comes up with a design, starts coding (either by himself or with his friends). If the project is of significant complexity, the first release probably won't do everything; it will have the basic features, and clear markers placed to indicate where future enhancements will go. But it must be useful as it stands, or else there isn't much point to releasing it; people would just look at it, say, "This is useless," and dismiss it. So, having created this usable first release, Joe releases the program with its source and announces that he's interested in finding people to help improve it. If enough people find the program worthwhile, and like the implementation enough to want to work with it, then he'll get the help he's looking for.

In other words, before you can expect people to want to improve your product, you have to have something worth improving. Additionally, you have to establish yourself as a credible programmer and project leader.

Viewed this way, one could infer that open-source is not really about creating new products, but about improving ones that already exist. And I think for the most part that that's true. Probably someone very well known in the OS community, such as Miguel de Icaza or Richard Stallman, could simply announce, "I want to start a new project around this great idea," and get lots of volunteers, but their fame and track records are, shall we say, atypical. Most of us have to prove ourselves first by doing the initial implementation, and I think that's a good thing, overall.

You need some source to open (4.66 / 12) (#4)
by Aquarius on Mon Oct 09, 2000 at 12:45:18 PM EST

ESR addresses the point of when a project should be open-sourced in The Cathedral and the Bazaar. His key point is:
Early reviewers and test audiences for this paper consistently raised questions about the preconditions for successful bazaar-style development, including both the qualifications of the project leader and the state of code at the time one goes public and starts to try to build a co-developer community.

It's fairly clear that one cannot code from the ground up in bazaar style [IN]. One can test, debug and improve in bazaar style, but it would be very hard to originate a project in bazaar mode. Linus didn't try it. I didn't either. Your nascent developer community needs to have something runnable and testable to play with.

When you start community-building, what you need to be able to present is a plausible promise. Your program doesn't have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is (a) run, and (b) convince potential co-developers that it can be evolved into something really neat in the foreseeable future.
(Chapter 9. Necessary Preconditions for the Bazaar Style, tCatB)

As has been pointed out elsewhere in this thread, attempting to initiate a project in bazaar mode very often results in a pretty web page, a pretty logo, ideas, and no code. This is not a good thing, and it's giving open-source as a concept a bad name.[1]

My advice: write something yourself first. Make it work. It doesn't have to work well, but it does have to present the "plausible promise" that ESR mentions above. If I wanted a program that did what your program will do (and that's the area where you'll get your co-developers), then I'll see what you've done and think "ah, it does that, excellent! But I'd like it to do this as well..." and -- pow! -- you've got a co-developer. If you say "I'm writing this, everyone pitch in at this stage" then you'll never get anything off the ground. People will bugfix broken software, marvellous. But inception by committee just plain doesn't work, in every example of it I've seen.

Aq.

[1] hm...idea for an article submission, there, once I've done a bit more research...

"The grand plan that is Aquarius proceeds apace" -- Ronin, Frank Miller
open source != other people do your work (3.00 / 8) (#5)
by recursive on Mon Oct 09, 2000 at 12:50:27 PM EST

Working in academia most of my software is available under a BSD style license. Although I can see from server logs that people at least take a look at my software I receive very few feedback. This is a common observation among people who share their code with others. The few project you read about constantly at That Other Site are the exception, not the norm. So do it for your self, share your code, and welcome any feedback you receive.

-- My other car is a cdr.


Re: open source != other people do your work (none / 0) (#9)
by Emacs on Tue Oct 10, 2000 at 12:05:17 PM EST

My experience is pretty much the same. My project is at the stage where it's functional for many people, and I see that people are downloading it or at least viewing my website, but I recieve very little feedback.

I have to wonder if this is because most people feel like they get this for free so they don't want to "bother" me, or they don't use the software, or they use it and have no problems or suggestions. Whatever the case may be, most people never offer anything.

I think the bottom line for an open source project is defining your goals and expectations, then living with them. My goal was to write something functional and to "give something back" which I feel I'm accomplishing. Since my goal was never to work with a team of hackers, or obtain fame and fortune, I have never been too concerned with the lack of interation with the rest of the world with regards to my project, not that it wouldn't be nice, but I'm not concerned with it.

[ Parent ]
We're employing a different approach. (1.50 / 12) (#6)
by psicE on Mon Oct 09, 2000 at 12:57:14 PM EST

My project, a game called "Blazing Blade" (not my idea), is under the BSD license. The source is downloadable now. However, no one knows about it, or how it works, so I don't have to worry about anyone spoiling it.

An Open Source Roadmap (3.00 / 2) (#7)
by Caranguejeira on Mon Oct 09, 2000 at 05:01:01 PM EST

In the Cathedral and the Bazaar, Eric Raymond compares the Open Source development model to a clamorous bazaar. If you actually attended a real bazaar, you might notice that the affairs at a given booth are more cathedral-esque than you imagined.

It is very valuable to have a roadmap for any Open Source project before you announce it to the world. In fact, it's good to have some code to show for it too (but not too much code; that could hamper the entry of new developers).

Your initial offering should be small, but functional. It should demonstrate that you are serious about the project. You should provide your vision of the future revisions of the program to any developers who might be interested in joining your team. This becomes your roadmap, and can be used to judge the quality of the project and the fitness of its code for future development.

Don't just provide vague items like "this program will someday support ... technology." Instead, cite a specific goal for each stable release. This will help solidify the development effort so that you aren't getting seemingly random patches for strange features. This also provides information for discussion that will allow your peers to refine your requirements for the greatest success in development and release. Developers feel like they have direction and are more inclined to work on the project.

You need a good web site where you can coordinate your efforts, disseminate information, and inform the public of your progress. There should be easy access to your source code. People should be able to figure out just what your project is about.

CVS access is definitely necessary for a project of any size. If you plan on having concurrent development, you will want source control. You might want to read "Open Source Development with CVS," by Karl Fogel, published by CoriolisOpen.

You should code to a standard. I recommend a visit to the FSF web site, and downloading the GNU coding standards. Also read "Developing software with GNU." Developers will be more comfortable with code that is consistent. Users will be happier if they don't have to bend over backwards to compile and use your program. Managing your project in accordance to standards will ease the development.

Once you have the management infrastructure to support your development (CVS, website) and some initial code (coded and built to standards), then you can start publisizing your project. Freshmeat is a good place to start. Join discussions and talk about it. Put it in your sig.

You will have a hard time getting people enthused about a vaporous Open Source project. I've mentioned some past projects in forums, got some response, and then it fizzled out. We had nowhere to start.

As the project maintainer, or Benevolent Dictator, you can't maintain your authority if you aren't doing the work.



How does anything get started? (3.00 / 2) (#8)
by AdamJ on Mon Oct 09, 2000 at 07:35:43 PM EST

So does the adage "Too many cooks spoil the stew" apply to open-source projects in the beginning phase?

I've found this to be true with pretty much any collaborative project that I've worked on, whether it be Open Source software, closed source software, writing, or Open Source writing. Projects that have had a small core of people with definite goals and organization have gone smoother and farther than those with plenty of people and plenty of ideas, but nobody willing to take the lead and set goals.

In the same vein, if you already have some code (or articles - I'm more interested in the written word than the coded word...) finished or in draft state, it's much easier for everyone to see what your intentions are and for them to figure out if what they have in mind fits in with your project or not, or if they're better off spending their time elsewhere.

Adam

Have a plan & at LEAST rudimentary implementation (2.00 / 1) (#10)
by wegster on Tue Oct 10, 2000 at 02:45:47 PM EST

Take a good look at some of the open source projects out there, on Sourceforge or wherever, and you'll likely notice something- a. A great deal of them are in 'limbo' without any useful work being done. b. Others look to be growing so large as to wonder if they'll ever be 'finished.' In 'another life,' I used to write MUDs, with up to 50 or so people _actively_ contributing. One or two worked out well, and others didn't. The best successes seemed to be when we'd done some serious thinking beforehand, wrote up specs and standards, along with a set of milestones. We(a few core people, generally 3-4 people) did the initial implementation, updated our plans and docs as we went, and then slowly accepted other people in different capacities, although it still remained up to the core group as to whether or not anything was implemented, or if someone's code became accepted into the whole. Think about Linus reviewing kernel patches here, we actually went through code line by line in most cases. While a bit anal, and didn't always make people feel overly 'trusted,' it worked. We made a plan, and stuck to it. As new people came on board, we adjusted the plan as you normally do in any sort of iterative development(IS there any other kind that succeeds??) as new ideas and considerations came into effect, but always steered things in the right direction. I also had the misfortune of working on 'less organized' projects, where basically everyone did as they pleased, with VERY little planning or coordination, and even less guidance on the overall project. Total insanity ensued- code quickly became unreadable, people were working at odds with each other, and it became questionable as to if things would ever manage to work right again. Not good. So, what's this boil down to? A few things: 1. You need a good and solid plan for whatever it is you're trying to do. Treat it like any other project, and do a set of specifications. This isn't a bad time to open the specs up for feedback, depending on the nature of your project. When you(original/core team) are comfortable with the specs, move on and do a real plan, whether an official type design doc, a simpler API doc, or a set of documents for each subsection in the project. Again, open it up for feedback if you like, just realize that YOU have the final word at this point. 2. Choose an appropriate source control tool. Yeah yeah, open source project, so everyone uses CVS. Your call..but might be worth making sure everyone can actually USE CVS properly...do a web page with configuration info, you can use it later. Think about bugs because you know you WILL have them, especially if you wind up with 50 people working on the project down the road.. 3. Do a basic implementation. Make sure your API will work for the project's needs. Adjust as required. How much work you do here largely depends on the scope of the project, number of developers in core team, etc etc..but stick to your plan! 4. Complete the set of docs that are 'required' for any new team members- if you've got specific coding styles/standards you're expecting to see and already using, document them. Set up a website if you like, and post them for all to see. Set up a mailing list for developers if it's going to be a project with more than say half a dozen people working on it. 5. More coding or 'open it up'? Depends. Your call. Check out your existing plan and docs, go over them with your group(if appropriate)...any large scale changes need to happen so far or in the foreseeable future? Figure out how many people you really NEED on the project, and try to break it down into subsystems, even if you haven't managed to work out the complete APIs as of yet. When ready, start 'advertising' and see what happens. Just remember you're a LOT better off with a half dozen programmers that work well together than 50 'experts' that can't play well with others, and don't stick to the plans. Once you do go ahead and 'open it up,' expect minor pandemonium to ensure at at least a few points in the cycle- it's inevitable, because you just added N new minds into the group, and each may have a new perspective and new ideas. Some of them may be worth revising your docs and plans for...Try to schedule an online meeting now and then(depending on project size, but try anywhere from a few times a week to bi-weekly) and log them. Make sure everyone discusses new ideas before implementing them, and at all costs, avoid the urge for feature creep. Get something that WORKS, ASAP, or you may easily become buried in a neverending cycle of "Hey, wouldn't it be cool if XXX, let me add it, it won't need to rewrite THAT much code!".. Get something out the door, even if it's only a CVS label and never goes to the public. And even when your plans change, stick to them ;-)

open source != collaborative (4.00 / 2) (#11)
by micco on Tue Oct 10, 2000 at 04:50:43 PM EST

Open source does not necessarily mean collaborative. If you read The Cathedral and the Bazaar closely, you'll see that ESR maintained the code pretty much alone. Occassionally others submitted bits of code (bug patches, etc.) but more often they just submitted suggestions which ESR implemented and released in the next version. He didn't have a team of coders working with him, at least for the most part.

I recently read an article (but lost the link...) which made this point very well. It pointed out that most of the famous open-source projects, linux in particular, have a close circle of coders who are "authorized" to change the code. Anyone can contribute, but all contributions go through this bottleneck in order to guarantee quality, interoperability, etc. In this respect, the "bazaar" model is actually quite cathedral-like.

Just to be clear about where I stand, I like the open-source model and think that most projects can benefit by enabling users to review the code. I would simply make the point that just because you open the source does not necessarily require you to make the project a collaborative effort and you can still maintain sole control over "official" updates. This means that all those other cooks can still make suggestions, but they don't get anywhere near your stew.

Also, just to be contrarian, you might read A Second Look at the Cathedral and the Bazaar for some reasons why open-source may not be the best model.

How does a Open Source Software Project get started? | 11 comments (11 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!