Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

In favor of picking apart the large open source projects

By Perpetual Newbie in Technology
Tue Feb 19, 2008 at 12:00:43 PM EST
Tags: linux, open source, programming, mozilla, open office, gnome (all tags)

I was reading LWN where there were reports of security holes in a dozen different packages based off Mozilla's Gecko. Someone complained about the lack of a universal library that they should all use so that users would only need to update the library and not each program separately. Apparently such a library exists as libxul, but its use is uncommon enough that the bugs were reported as being for the several separate programs and not for "libxul".

The Mozilla project, like Netscape before it, is a grand undertaking to kill the desktop by letting people run network-based applications on a portable browser platform. Since the existing open-source GUIs were not robust or featureful enough for the Mozilla developers' liking, they built their own. Mozilla now replicates much of what GUIs do, it does some of it better, and it is open source. Why aren't some of its components part of other open source GUIs? Why isn't libpr0n (or whatever Mozilla's image library is called today) now the standard image loading library for open-source GUI programs?

The various Apache projects include some nifty things that should probably be part of the common OS, such as mime type autodetection in httpd's mod_mime_magic. The unix "file" command does the same conceptual task. Both are even based on the same original code from the same author, Ian Darwin. Why aren't they sharing a common library?

Then there is OpenOffice. If there was ever a project that needed to be diced up into little pieces, this is it. Here you have one monolithic chunk of code that does everything its own way which is not the way anything else does anything.

Of all the big projects Gnome seems to be doing the best job of dividing their project into reusable components and offering them to developers as such. Unfortunately, it does not seem to have translated into popularity. Projects appear to either use Gnome, all of it, or do not take advantage of what it offers. Some of Gnome's components are useful whether you are doing GUI programming or not. Consider its file metadata system, a standard way of assigning arbitrary attributes to files. Something like this should be part of the standard Unix toolkit and there should be command line tools based off it.

Gnome also offers its own mime types system separate from Apache and from the file command.

Here is the bumper sticker version. Competition is good. Reinventing the wheel is bad unless your wheel is better. Having multiple different versions of the same thing is just wasteful.


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


Related Links
o Also by Perpetual Newbie

Display: Sort:
In favor of picking apart the large open source projects | 33 comments (25 topical, 8 editorial, 4 hidden)
Well (2.00 / 3) (#3)
by sausalito on Sun Feb 17, 2008 at 06:16:22 AM EST

Sometimes the project history is also to blame (see Openoffice). Alternatively, psonality clashes, egos and downright laziness are the culprit.

That's the almost inevitable result of open source: a number of independent projects not co-ordinated together each working on their own.

On an average Linux desktop you might find four GUI's: QT, GTK, Mozilla and Openoffice. It's inefficient, yes. But then again, memory is cheap, drive space too so it does not matter. Don't know if this is bad for security: after all it might be just as bad if you develop on a platform you don't know well and you don't feel responsibility for.

GBH - "The whole point is that the App Store acts as a firewall between busy soccer moms and goatse links"

Because they are open-source... (none / 1) (#24)
by jd on Mon Feb 18, 2008 at 05:08:17 PM EST

...anyone can hack patches for such programs to let them use code elsewhere for duplicated functions. Look at scientific software - heavy re-use of code, everywhere, with masses of interdependencies using dynamic libraries or even loadable shared objects. Brilliant.

The only danger in such a tactic is that function calls ARE expensive on time and memory, and system calls even more so. Computers do have more memory and power today, but don't waste what you have, or you're no better off.

A well-balanced, well-written program should be fast, have a very low footprint, and still re-use code whenever possible. It can be done. It has been done. Question is, will anyone writing anything mainstream do this?

[ Parent ]

The reason includes licensing incompatibility (3.00 / 4) (#4)
by ksandstr on Sun Feb 17, 2008 at 09:28:49 AM EST

The Mozilla project uses a licensing model that is incompatible with most Free Software and thus, the bulk of so-called "open source" as well. In addition the suite of tools produced by the Mozilla project are rightly seen as being of unsuitable quality for use in tasks that are more critical than web browsing, e.g. the viewing of pornography in a slideshow fashion (you wouldn't want your computer to run out of memory during the vinegar strokes, would you).

So yeah, no. Mozilla is large, ugly, slow, incompatible (the current C++ ABIs are inaccessible from C, the lowest common denominator) and memory leaks in the Mozilla suite are generally left untouched despite ample availability of the tools to hunt down and repair these defects. It is also governed by a corporate entity which exercises jealous trademark controlfreakery, and is thus seen as not playing well with Free Software.

That answer your question?


You hit on many of the main reasons (1.50 / 2) (#7)
by Perpetual Newbie on Sun Feb 17, 2008 at 01:53:05 PM EST

Not much can be done about code quality other than improving the code which takes time, skill, effort, money, etc. Incompatibility and project manager asshattery are not even that easily fixable.

I believe a lot of the problems with licensing could be addressable by promoting a developer culture where licensing is more fluid:

  • Project maintainers should consolidate copyright so that licensing can be changed at will
  • Hackers should grant projects dual copyright to their copyrightable patches so that both the person who wrote the new code and the person in control of the main project can do whatever they want with the code in the future
  • Projects should use a common GPL or GPL-compatible Free license for their main releases so that the code remains free, and be open to rereleasing their code under an alternate license for specialized needs (and for $$$ if it would create a closed/unwanted fork)
  • Reusable libraries should be split off from the main branch and LGPLed by default so that other projects can use them.

And if someone doesn't give a damn about licensing or other people stealing their code then they can just release to the public domain.

[ Parent ]
in favor of dumping this story (1.80 / 5) (#5)
by chlorus on Sun Feb 17, 2008 at 09:29:28 AM EST


I installed Debian the other day (2.92 / 14) (#6)
by Empedocles on Sun Feb 17, 2008 at 11:30:03 AM EST

Naturally, I assumed Firefox would be part of the default install, being open source and all that stuff the Debian guys like.

It wasn't. Not only was Firefox not included as part of the defualt install, the package manager (Synaptic) didn't even list Firefox as an available package. Instead, some piece of shit called Iceweasel was listed. Rather confused, I downloaded Firefox and installed it the old fashioned way.

I was rather confused as to why Debian, an open source zealots dream, didn't include one of the premier open source projects available. My interest piqued, I decided to research this travesty.

The reason a Firefox package wasn't included in Debian? The icons. That's right, the Debian guys didn't like the licensing for the icons included with Firefox, so they didn't include it. Instead, they created their own icons for Firefox, called it Iceweasel, and included that. Other than that, Iceweasel differed from Firefox in no way whatsoever.

The moral of this story? Debian developers are petty and stupid. They also enjoy shooting themselves in the foot.

And I think it's gonna be a long long time
'Till touch down brings me 'round again to find
I'm not the man they think I am at home

Iceweasel (2.57 / 7) (#8)
by Perpetual Newbie on Sun Feb 17, 2008 at 01:58:02 PM EST

I would not say "petty and stupid" but "idealistic". Well, calling it assweasel is petty but not so much the rest of the situation. The goal of Debian is to have a pure GPL-or-freeer distro and Mozilla's trademark policy crossed their line, even if ever so slightly so. I am still surprised they couldn't work something out.

[ Parent ]
obsessive-compulsive bureaucrats (3.00 / 2) (#19)
by Corwin06 on Mon Feb 18, 2008 at 08:07:21 AM EST

not idealists

when people take their idealism to pettiness and stupidity, it's called bureaucrat-level nitpicking

GPL or freer only, whine about license of icons. Or close your eyes on th problem because the fix os obviously much worse (cofusing users)

It's not like if Mozilla would ever sue Debian, yeah?

Thus they confuse their users with stupid fixes to non-issues

Petty and stupid

"and you sir, in an argument in a thread with a troll in a story no one is reading in a backwater website, you're a fucking genius
[ Parent ]
Surprised at your suprise (2.62 / 8) (#10)
by sausalito on Sun Feb 17, 2008 at 02:41:21 PM EST

You find this out now... this happened like two years ago. Ubuntu had the same problem in their first one or two versions, too.

The fact is that Mozilla's licence forbids any modified version of the software from being named "Firefox" and from using its art without the Mozilla Foundation approval. Of course, a volunteer organisation did not want to go through this useless papework, which is also against their perception of the "free software ethos" (RMS liberties etc).

It's Mozilla's fault, not Debian's. Mozilla under Mitchell Baker had grand plans of IPO and stuff. They wanted to keep control of the brand and did a lot of questionable things (like the licence choice, the creation of a for-profit parent company without releasing information on the financials of the group...).

I think that she was in purely for the money, and did a lot of damage. Luckily now she's gone.


GBH - "The whole point is that the App Store acts as a firewall between busy soccer moms and goatse links"
[ Parent ]

money- (1.50 / 2) (#11)
by nononoitaintmebabe on Sun Feb 17, 2008 at 02:53:38 PM EST

the root of pretty much all evil.

[ Parent ]
and of quite a few good things as well % (1.75 / 4) (#12)
by sausalito on Sun Feb 17, 2008 at 03:13:07 PM EST


GBH - "The whole point is that the App Store acts as a firewall between busy soccer moms and goatse links"
[ Parent ]

that's probably true, but since (2.00 / 2) (#13)
by nononoitaintmebabe on Sun Feb 17, 2008 at 03:29:40 PM EST

i'm poor and don't ever get to have those good things, then i like to think it's purely evil.  

[ Parent ]
the LOVE of money is the root... <nt> (none / 1) (#29)
by justo on Tue Feb 19, 2008 at 02:24:50 PM EST

[ Parent ]
I dunno (none / 1) (#31)
by Herring on Sat Feb 23, 2008 at 02:21:26 PM EST

The love of pre-teen girls is the root of some evil.

Say lol what again motherfucker, say lol what again, I dare you, no I double dare you
[ Parent ]
So you say money is the root of all evil? (none / 1) (#32)
by kurtmweber on Sun Feb 24, 2008 at 06:18:04 PM EST

Have you ever asked yourself, then, what is the root of money?

Let me give you a tip on a clue to mens' characters: the man who damns money has obtained it dishonorably; the man who respects it has earned it.

Kurt Weber
Any field of study can be considered 'complex' when it starts using Hebrew letters for symbols.--me
[ Parent ]

What can I say? (2.00 / 2) (#17)
by Empedocles on Sun Feb 17, 2008 at 10:49:29 PM EST

I've been using Slackware for forever and a day now. This useless open sores controversy is all new to me.

And I think it's gonna be a long long time
'Till touch down brings me 'round again to find
I'm not the man they think I am at home

[ Parent ]
it wasn't just the icons (3.00 / 10) (#14)
by Delirium on Sun Feb 17, 2008 at 03:40:01 PM EST

They were the initial issue, but the bigger stumbling block is that the Mozilla Foundation requires any modifications—patches, customizations, whatever—to be approved in advance by the Foundation before anyone else is licensed to use the "Firefox" trademark. Debian's policy is not to agree to such agreements—this being the free-software world, they want the freedom to make their own modifications, patches, etc. without advance permission.

[ Parent ]
-1 Pointless. (2.33 / 3) (#15)
by The Amazing Idiot on Sun Feb 17, 2008 at 08:19:57 PM EST

Lets go fly a kite.

Up to the highest height! (none / 1) (#22)
by rpresser on Mon Feb 18, 2008 at 11:16:11 AM EST

Let's go fly a kite
And send -- it -- soar -- ing
Up through the atmosphere
Up where the air is clear
Oh, let's go fly a kite!
"In terms of both hyperbolic overreaching and eventual wrongness, the Permanent [Republican] Majority has set a new, and truly difficult to beat, standard." --rusty
[ Parent ]
Why it isn't used more... (2.25 / 4) (#20)
by boxed on Mon Feb 18, 2008 at 09:49:20 AM EST

Simple: because it sucks. Am I talking about libxul or some other thing you mentioned you might ask? Yes would be the reply. I am talking about all your examples.

The truth of the matter is that open sores is very rarely anything other than crap. Just like commercial software in fact.

What the fuck is this shit? (1.85 / 7) (#27)
by daveybaby on Tue Feb 19, 2008 at 04:55:15 AM EST

Half-arsed drivel about open source, which asks a few inane questions and whines about the state of things yet fails to provide any insightful analysis or answers.

And it has positive score?
Sweet fucking potato christ this place is fucking fucked to fuck.

Is voting this shit up some kind of new troll? If so, i have bitten.

look at the first story on the front page. (2.50 / 4) (#28)
by Morally Inflexible on Tue Feb 19, 2008 at 11:59:09 AM EST

we'll vote any crap up.

[ Parent ]
Ok, a rundown on my thoughts, as if anyone cared. (none / 1) (#33)
by jd on Tue Mar 18, 2008 at 02:52:00 PM EST

  • Reusable components are good
  • Reusable components should be as low-level in the system as makes sense and no lower
  • Inventing new wheels is not a problem, if the axle is a standard - likewise, replacing a software component is effortless if there's a standard way of making use of it
  • There are many Open Source GUIs - Berlin/Fresco is one that is amazing - it is the users, not the developers, who have kept X11 alive
  • There are many toolkits for profiling software, detecting memory leaks, analyzing resource usage and debugging at the source level. It's unreasonable to ask most users to use these tools or submit in-depth studies on source code to developers, but how many of those users who COULD do this actually do?
  • How many frameworks (eg: KDE, Gnome, Motif, FLTK, Afterstep, WindowMaker) provide mechanisms for non-expert users to submit bug reports to the right developers if a third-party application using that framework crashes? Of those that do, how often are the bug reports actually any use?

Suggested actions for framework developers:

  • Develop better bug-reporting tools.

    • On initial connection to the framework libraries, an application should be able to register a point of contact for e-mailed debug information to go to.
    • On initial connection to the framework libraries, an application should be able to register a bug-tracker system (type, URI, and layout of report) for directly filed debug information to go to.
    • The framework should be able to record a session of an application. If an application crashes, the user should be offered the ability to automatically re-run that session in a special debug or profiling mode, so that developers can get extra information on what went wrong.
    • If a bug-tracker is registered, then all tracker numbers for bug reports to that tracker should be recorded and should also be erasable by the user.

  • Provide extra debugging/profiling code, such that when distros build debug and profile versions of the code, there is enough information to be useful.
  • Provide mechanisms by which a bug-tracker can be queried.

Suggested actions for application developers:

  • Make as much use of debug/profiling hooks as possible in the supported framework(s).
  • If automatic bug reporting is supported, make sure that both the feature and the bug reports are used.
  • QA the code using unit tests and integrated tests as far as is reasonable. Don't assume that because the code compiles that it works.
  • Run scanners over the code, such as Valgrind.
  • Link the code to debugger utilities such as ElectricFence or dmalloc (in the case of bounds checking and memory leak detection) and make sure that the code isn't doing something strange.
  • Add extra conditional code blocks for when compiling in debug or profile modes to preserve useful information.
  • Provide mechanisms by which a user can list closed and still open bugs for that application.

Suggested actions for users:

  • Avoid, where possible, frameworks and applications that impede debugging and development. (There's no point complaining about code not working if you can't say anything useful and developers can't do anything with what you do say anyway.)
  • Initial bug reports should be from the code in its regular state. This information is inherently very limited, but is the only information users can provide in the case of both unrepeatable bugs and Heisenbugs.
  • Users should provide developers with the best information they can.

    • If a crashed session has been logged, they should find time to replay that session under as close to the same conditions as possible, but with debugging or profiling enabled.
    • If the bug is repeatable, they need to file the automatically collected data and a comment about the circumstances surrounding the crash. In the case of bug-trackers, this should generally be automatically inserted as a follow-up on the original (non-debug) report.

  • The user, on discovering bugs in the software, is ultimately responsible for ensuring that the software they use does what they want. If critical (to the user) bugs aren't getting fixed, the user must determine if this means they need to supply additional information, switch packages, or approach things differently.
  • The user should recognize that they have no business complaining about bugs they don't supply enough information on for the developer to fix, provided the user is given a means to supply that information.
  • The user should complain when genuine, repeatable, adequately detailed bug reports are not handled effectively within a reasonable time. (Reasonable is a function of the resources available. Single developer projects can't fix things in the same timeframe as projects with many active hands.)

In favor of picking apart the large open source projects | 33 comments (25 topical, 8 editorial, 4 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!