Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

[P]
Where has all the thought gone?

By mind21_98 in Technology
Sun Nov 26, 2000 at 06:56:04 AM EST
Tags: etc (all tags)
/etc

This afternoon I was writing a C program that would send a newsletter to all subscribers. The script was for a client at work. After the script was done with, I looked at the code. I realized that in it was the basis for a MLM such as Mailman. Unfortunately, I realized that many people have already made excellent MLM software; a few were involved in production settings.

After seeing this, I began to think: what has happened to software innovation?


It seems that for a while now, software was going into a downward trend. I've noticed that no new genres of software have been created. However there were improvements made on certain bits of software, enough for the potential customer to take another look and possibly buy or download it again. This means that thought in software is not dead, at least not yet. I have several reasons why software development, at least in the non-gaming arena, is not where it should be.

Lack of Need

One possibility is that there is currently no need for any new genres for the average user. They are probably sasitified with the "status quo", i.e. word processing and spreadsheets, and maybe the occasional web browser or two. As long as they have those types of applications and as long as they work, the average user is happy.

Lack of Ability

For the other, more specialized users, they want something more than just a word processor and a Web browser. They might need specialized applications for their line of work. Most of them can not afford custom programming, so they cannot get someone to write them a program that suits their needs. Only a few of these specialized people know how to program as well. Therefore, many of these people cannot get the applications they need to perform their work.

Both of these aspects combine to make development for specialized fields much more expensive and scarce than for common applications like word processors. The way to improve this situation is to make software development more open. More people should have direct relationships to the developers, to suggest things that should be added to a particular work to make it more useful. Without these things there would be no way to tell the developers out there what the users demand, resulting in software that reflects the interests of the company who developed it, not of the users that will eventually install and use the application. Hopefully it will become possible to make developers put a little more thought before coding.

Sponsors

Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure

Login

Related Links
o Also by mind21_98


Display: Sort:
Where has all the thought gone? | 23 comments (23 topical, editorial, 0 hidden)
proprietary software (2.62 / 16) (#1)
by enterfornone on Sat Nov 25, 2000 at 09:29:25 PM EST

Proprietary software forces programmers to reimplement software that already exists. If all software was free programmers would be free to innovate and adapt it.

People say that you can't make money out of free software but what they don't realise is that only a few companies make money from software, while all companies use software. Many companies pay for software yet still have to employ programmers to intergrate it into their systems.

If companies could obtain software gratis and modify it to suit their needs, it would give their programmers more time to innovate.

--
efn 26/m/syd
Will sponsor new accounts for porn.
*sigh* Software Innovation Has Always Been Slow (4.43 / 16) (#2)
by Carnage4Life on Sat Nov 25, 2000 at 10:17:28 PM EST

It seems that for a while now, software was going into a downward trend.

Really? Downward trend from when? 1990? 1980? 1975? 1998?

I've noticed that no new genres of software have been created.

Can you please define 'genre of software' or at least show a period when more new 'genres of software' were being produced than today?

Progress in the software industry has always been relatively slow. I've expounded on this in an earlier thread so I'll just create some new points of reference with this post. It takes years for theories and research to reach critical mass (e.g. Hypertext has been around since the '60s, Ethernet since the '70s, SGML since the '80s, relational database theory since the '70s).

New paradigms in computer science are being created all the time, DNA computing, Quantum Computing, A.I. research and genetic algorithms, Object Oriented database technology, various distributed component technologies, advanced microkernel research, heck the web browser is only a few years old, simply because consumers aren't downloading product X with whizzbang new features every year does not mean computer science/software is stagnating but is instead business as usual.



Hmm... yes and no... (3.25 / 8) (#3)
by bgalehouse on Sat Nov 25, 2000 at 11:05:55 PM EST

Yes in that, in many fields, technology is only widly deployed well after it's initial creation.

Most people haven't been hearing about Java for very many years, but all the pieces which went into it have been around for quite a while. Possibly excepting bytecode verification - I don't know of a research system which uses it for much. Everything else had been done and debugged before.

No, in that, sometimes cool tech catches on as it comes out.

I don't think that Google's graph analysis is something that just anybody could code up in a weekend. But it didn't take long for Google to catch on. Similarly, the Quake/Doom computational geometry tricks played by ID are not trivial, but, AFAIK, were novel at the time and caught on quickly.

[ Parent ]

it depends on what you mean by "slow" (3.80 / 5) (#13)
by SEAL on Sun Nov 26, 2000 at 01:43:45 PM EST

There's always a lag time between discovery and implementation. Many algorithms and ideas have been known, but not applied in the computer science community for a long time. Let me throw out a few examples:
  • discrete cosine transform / Fourier analysis. Old stuff, finding new applications in sound and video compression.
  • BSP trees, as you mentioned, used in Quake and descendants.
  • The A* pathfinding algorithm. It was developed back in 1968. Only recently, it has become the standard for computer AI in games, especially real time strategy variants.
  • Quaternions. More old stuff - finding use in modern 3D engines, because they have some benefits for handling rotations.

I'm sure there are many more. Obviously my view is tinted by my profession (game programmer) :) Part of the reason for the delay in application, is that many programmers don't have the formal background to even know these things exist. Other times, we run into a situation where the idea is good, but the hardware hasn't caught up yet. And finally, you get the real gems: a completely new use - for example, A* saw use in WAN pathing long before games.

Best regards,

SEAL

It's only after we've lost everything that we're free to do anything.
[ Parent ]

Like maths (4.50 / 2) (#19)
by Simon Kinahan on Mon Nov 27, 2000 at 07:05:00 AM EST

It strikes me that this is very similar to the relationship between maths and applied maths. It could in fact be argued that its exactly the same relationship, as many of the algorthms you mention were developed in mathematics, not computer science.

Almost all mathematical discoveries were explored for their academic interest and sheer elegance before being applied. Complex numbers, quarternions and boolean algebra were all interesting and yet "impractical" until someone found their application to real-world systems and they became the basis for modelling all kinds of things. There are exceptions, of course, like calculus or matrices that started out as solutions to pressing real world problems and only later developed into fine mathematical abstractions. I suppose the same thing has happened with algorithms, but actually I cannot think of an example right now.

Simon

If you disagree, post, don't moderate
[ Parent ]
I agree (none / 0) (#21)
by SEAL on Mon Nov 27, 2000 at 05:09:14 PM EST

I didn't mean to exclusively point my post towards computer science: rather, that's just my area of expertise. Computer science is an applied science, like mechanical or electrical engineering, and so on. These disciplines depend on mathematics for their foundation, and for problem solving.

This is yet another reason why we in the U.S. need to stop letting the math curriculum slide in our schools. When kids enter college without a solid foundation in mathematics, it closes many doors to them, or at least forces them to play catch-up. Of course only a few will pursue a pure mathematics degree, but math influences many other areas of interest. Even outside of engineering (economics, anyone?).

Algorithms are usually developed as a means of solving a specific problem. So in that sense, they are prevalent in the computer science community. However, it is mathematics that can prove the complexity of an algorithm. Computer science may provide the method, but math can tell you if it's worthwhile :)

Best regards,

SEAL

It's only after we've lost everything that we're free to do anything.
[ Parent ]
The programming of the future is all glue. (3.66 / 9) (#4)
by porkchop_d_clown on Sat Nov 25, 2000 at 11:22:42 PM EST

New ideas are rare in any field, especially once it has reached stage of maturity. For software development, this means that there's going to be less and less original development work going on, and more and more integration work. I've seen it at many companies - my current employer has finished switching from being a product company to a "solutions" (i.e., consulting) company. As part of my job, I'm often forced to make existing software work in ways it wasn't designed for - when I know I could write something better from scratch. But simple economics dictates this approach - economics of money, but also economics of time. As the world of "What Is" gets larger the world of "What Isn't" gets smaller, and the customer isn't going to pay for 6 months of my time and be my guinea pig so I can prove what a studly programmer I am, when in 3 months I can reasonably glue together existing packages to do the job they want.

Someday, someone will write the replacement for Linux. I have no idea what it will look like. But I'm absolutely certain the poor b***ards will spend quite a lot of time making sure their new OS is POSIX and X11 compliant - so that it can run EMACS and XTerm.



People who think "clown" is an insult have never met any.
True but.... (3.25 / 4) (#11)
by Khedak on Sun Nov 26, 2000 at 11:05:22 AM EST

This is true if the rest of technology remains stagnant, but there aren't many technologies that have remained stagnant for long. For example, agriculture is one of the oldest known fields. Agricultural technology however has continued to innovate. Why? Because other fields have innovated, and those innovations have been applied to agriculture (e.g., advanced genetics, chemical and biological pest control, etc.). It's true that if nothing comes in from other fields, our field will eventually stagnate. But when innovation falters in a field, innovation can be applied from elsewhere.

Essentially this is the same as saying that either software will stop innovating, in which case other fields will continue forward and surpass software to such a degree that we won't need software anymore, or that software will continue to innovate and advance along with the rest of technology. Technology has quite a long way to go. And I don't mean in a goal-oriented sense. Technology, like evolution, isn't goal-directed, but is "selected" through usefulness and appeal. Neither biological nor technical innovation is going to stop any time soon.

[ Parent ]
quit innovating for a minute (3.90 / 10) (#5)
by beertopia on Sun Nov 26, 2000 at 04:19:10 AM EST

... and make the stuff we've already got work right.

Err, maybe that would be innovation. I'm semi-serious here, but the previous posters have already made the serious points better than I could... the web browser's a good example. It's a fairly new sort of program, too new to be, like, old hat already or anything. But, Netscape and Microsoft started "innovating" it all to hell and gone, without ever making it do its basic job reliably, and every one they bring out now, they've put their time into adding all sorts of crap that either programmers or marketers (or, I dunno, maybe even users) thought sounded nifty-- so they're feature-rich bug-ridden crash-a-holic pieces of crap.

But, so, on the other hand, p2p stuff for example is HOT HOT HOT right now, brand new paradigm, blah blah blah- it may be overhyped, but there's a lot of smart people trying to make something interesting out of it. Isn't that innovating? I mean, I don't know, I'm asking.

And, on the third hand, what's an example of a brand-new genre of software that would be useful (or even just cool)?

And what is innovation? (3.75 / 8) (#6)
by Aquarius on Sun Nov 26, 2000 at 04:29:43 AM EST

You see, this question depends on what you mean by "innovation". The difference is shown up in your answer to the question: Was Tim Berners-Lee's creation of HTTP/HTML innovative? What about James Gosling's creation of Java?

You see, the idea of hypertext, and the idea of virtual machines running bytecode, are not new ideas. What the above-named did was to make a killer app for those technologies. So, who's innovative? The person who thought up the idea in the first place? Or the person who makes the killer app?

If your answer is (a), then I'll bet you there's innovation by your definition going on all the time, it's just that you haven't seen it because these things languish in obscurity until someone writes a killer app; that's inherent in the very nature of killer-app-ness, I think. If your answer's (b), then it'd be interesting to see when you think innovation stopped. 1980? 1990? 1995? 1999? Last Tuesday?

I think, from what I'm reading into your post (and do correct me if I'm wrong), that what you're bemoaning is the lack of new application types, not the lack of innovation within applications. For instance, yes, writing a word processor is not per se innovative, because there already are word processors. However, Word 2000 is a lot, lot different from WordPerfect 5.1. Look at the advance in things like component technology, where your humble, it-already-exists,-<yawn> word processor can be spawned within other windows; dialog boxes can have full access to a WP's editing power; they have built-in scripting languages, can be controlled from other applications, can communicate with XML or SOAP or any one of a dozen other things. Why is this not innovation? Just because someone hasn't come up with a new type of app doesn't mean that they're not innovating; developers are creating new ways to do the things we already do in a cleverer or quicker or more efficient or prettier fashion.

Besides, innovation is, er, hard. I don't necessarily think that it's something to be produced on demand. What you do is make the most of it when it comes; you wait for a Richard Feynman or an Isaac Newton or an Einstein to come along with a wild idea, and then the rest of the world consolidates that idea into a real thing. Too much innovation leaves you with software that never makes it out of alpha and crashes all the time :-)

Aq.

"The grand plan that is Aquarius proceeds apace" -- Ronin, Frank Miller
Wait and see (3.40 / 5) (#7)
by dreamfish on Sun Nov 26, 2000 at 06:19:49 AM EST

When you talk about there being no new innovation in software genres it reminds me so much of the Patent Office at the beginning of the 20th Century saying that 'Everything that can be invented has been invented'.

Innovation doesn't just cover the big things, like spreadsheets, but little things like algorithms, optimisation techniques, protocols, etc. Admittedly one 'big' genre I can think of that has seen major innovation over recent years is business intelligence systems used for data mining (like in analysing consumer shopping trends).

I like to think of it in the following way: have we already got all the technology and software to create the computer systems on Star Trek? No? Then a lot more innovation is required :) It's just that people are working on this but they're taking their time (often years) and will only release it when it's in production-quality. Not everyone releases early and often :)

I don't belive your solution (2.25 / 8) (#8)
by boxed on Sun Nov 26, 2000 at 06:55:55 AM EST

The solution is in my eyes to have programmers employed by public means to devolop these programs that aren't commercially viable but that would make our society more efficient.

And I don't beleive YOUR solution... (2.80 / 5) (#14)
by daystar on Sun Nov 26, 2000 at 02:05:29 PM EST


Socialist programming is going to be better? You base this on what, the massively successful soviet software industry?


--
There is no God, and I am his prophet.
[ Parent ]
Russian hackers (2.66 / 3) (#17)
by h2odragon on Mon Nov 27, 2000 at 05:06:59 AM EST

...not that I disagree or anything, but there do seem to be more Russians involved in the deep head cracking stuff than in the rest of the industry, proportionally.

[ Parent ]
Or... (none / 0) (#20)
by Mr. Neutron on Mon Nov 27, 2000 at 01:48:43 PM EST

Maybe they're just more visible. Either because they get caught more often (ie, they're bad "crackers"), or they pull off greater stunts more often (that is, they're either intelligent and/or ballsy).



[ Parent ]
haha (1.00 / 1) (#18)
by boxed on Mon Nov 27, 2000 at 05:42:50 AM EST

Comparing ANYTHING to the soviets is really silly. They tried their hardest to make the worst out of everything they had and they succeeded pretty good. They didn't have computers out in the public so naturally the programming they did would be quite different.

[ Parent ]
No new genres? (3.00 / 7) (#9)
by joto on Sun Nov 26, 2000 at 09:11:07 AM EST

Let's see if we can dig up some new genres of software in recent years?
  • The World Wide Web (The web itself is not only a new genre, but creates new genres all the time)
  • Java, new Amiga, .NET (All interesting new twists on old ideas. We did not have this type of virtual machines earlier)
  • Component technology (CORBA, COM, ...)
  • Audio and video compression
  • Video editing
  • Virtual reality environments
  • XML and all it's associated technologies (not just for the Web)
  • Application scripting languages (not exactly new, but only lately it has become mainstream)
  • Parental filtering software (yes, it's a new genre, although stupid)
  • Practical speech recognition (mostly hype for now...)
  • New and interesting programming languages (especially in the functional and logic programming world)


Bleeding edges (3.28 / 7) (#10)
by slaytanic killer on Sun Nov 26, 2000 at 10:29:21 AM EST

You know, this commentary brings me to the thought that people can really be so much into one mode of thought, that they fail to see orthogonal innovations.

This really is the golden age of computing. The areas one can get into are bewildering. If you want something cutting-edge on the foundational level, go develop with the HURD people. Or make one of those old languages like LISP/Scheme finally glow by using it with flexibility that leaves all the other languages in the dust. There are so many old ideas that no one has had time to pick up yet, in the race to find new ones. (Usually, a race to beat patents.) Theorize about quantum machines, their limits and possibilities.

Now, from a consumer's perspective, it is understandable to have this perception of "nothing new." Consumers are notoriously jaded, once they begin to understand the bounds of something. Even Turing knew about the fun & inescapable bugs of programming, as he was postulating the machines; there are certain bounds one can deduce a priori. But someone from inside of the core can see all the subtleties missed by outside observers.

Depends on what 'new' means (2.85 / 7) (#12)
by gregholmes on Sun Nov 26, 2000 at 11:09:08 AM EST

Today's automobile is just an internal combustion engine propelling a chassis down the road. Very old hat. Yet you can't really deny there has been lots of innovation.

And regardless of whether the programming ideas are new, something new is created when new value enters the world. If I can get free software, that I can modify, for a purpose I had to get $900 software closed-source before, then there is something "new" there.

As for lack of ability among the users to write their own, I think more free software developers should do the hard thing and develop free software that is needed, rather than cool. Shameless self-promotion - I am trying to do that.



What are you talking about? (3.75 / 8) (#15)
by delmoi on Sun Nov 26, 2000 at 03:00:03 PM EST

I honestly have no idea what you're talking about. I mean, yeh there isn't going to be much innovation in Unix (other then cool XML stuff :P) command line tools, but new kinds of software are being developed all the time. Have you missed all the debate about napster? Do you know what gnutella is? Were there any XML parsers 10 years ago?

There is huge development right now in all kinds of distributed computing models. I mean, geez, in just the past 5 or 6 years huge advances have been made in plain old HTTP/HTML. And now we have all sorts of things like RPC over the Internet using standards like SOAP on java.

Just a few years ago we got consumer software that would take dictation nearly perfectly (it may just have been the point where consumer level CPUs were powerful enough)

I suppose the average user doesn’t need more then spreadsheet+wordprocessor+web+email, but those have been around for a long time. If you take out the web browser, more then 20 years. But that doesn’t mean that there is no more software innovation, just not so much on the client side.

The fact is, you're spending you're time reinventing the wheel. I don't know why you felt like doing something that's already been done, but whatever you're reason was, it doesn’t really affect everything else.

Go out and do something innovative. Read up on Java, and XML, and distributed computing, and all that other stuff you seem to be unaware of (or didn't think about) and make something innovative.
--
"'argumentation' is not a word, idiot." -- thelizman
On the other hand... (3.60 / 5) (#16)
by mwright on Sun Nov 26, 2000 at 06:52:02 PM EST

Yes, there are some major developments; however, in some areas, it's been disappointing.

For example, 5-10 years ago, the variety in computer games was quite amazing. I look back now at all of the games that I played, and I realize that there really were no two that I had that were alike (with the exception of a later episode, in some cases).

Today, generally, when I hear about a new game, my thoughts are "Yay. Another first person shooter" or "Another real-time strategy game" or such. It seems like, today, game makers stick to a few types of games that are known to make them money, and just try to make one with more levels/guns/units etc. than the others.

[ Parent ]
Programmers in Linux are new. (none / 0) (#22)
by matman on Wed Nov 29, 2000 at 02:59:38 AM EST

Linux is new, and it's popular; it draws a lot of attention from young geeks. These new geeks are learning to program; most small projects (the majority of projects) are lead by programmers that have not written many more than a few medium sized apps. These programmers don't have the experience or the confidence to see where oportunities exist, or how to implement all of their new ideas. This makes a lot of new software in Linux not-so-innovative, and more reinvention of the wheel.

But that's alright. The new, idealized programmers will grow up and make new, original and ground breaking software - hopefully :)

Separating the wood from the trees (none / 0) (#23)
by tumeric on Fri Dec 01, 2000 at 01:40:36 PM EST

Try visiting freshmeat daily -- there is plenty of innovation among the repetition.

Where has all the thought gone? | 23 comments (23 topical, 0 editorial, 0 hidden)
Display: Sort:

kuro5hin.org

[XML]
All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!