1. Directions in System Development.
I have identified 4 key technological innovations which represent the kernels of the new building blocks that will eventually build the massive IT systems of the future.
- Reiser File System which is just the tip of the iceburg in terms of implementing name spaces and highly efficient next generation file systems.
- Plan 9 which really does unified naming in a big way. Its open sourcing will start to push this more into the mainstream as people get a look at the implementations and wrap themselves around the concept. True unified naming is the Holy Grail for system design, whether system architects yet realize it or not ;o)
- Transmeta's CMS system The Transmeta code morphing system and associated innovations in hardware design harald a truely new era in the design of systems and their relationship with software. I predict that in a few years the concept of a "binary" is going to start getting pretty fuzzy...
- XSL and XSLT Systems like Cocoon represent the dataprocessing environments of the future. XSLT has some very important properties, mainly the fact it is ITSELF XML, which means that XML can describe how to process itself! This brings XML itself to status of "logically complete" system. And at a rather high level at that.
2. What are these things.
Reiserfs is a Linux file system which allows the efficient storage of massive numbers of small files in very large and very deep directory structures. This allows the pushing of the existing file naming schema down to the level of individual records, removing lots of logic from applications. (IE, what if you just had to open the file /home/jsmith/contactlist/joeblogs/phonenumber to get at Joe Blog's phone number, instead of writing a program that opens the contact list file and searches it for that information? Can you see how much less logic this requires? You can do that now, but if you had 10k records in contactlist you would have a horribly inefficient mess under existing file systems. Reiserfs fixes that problem.
Plan 9 is an OS based on Unix and developed by Dennis Ritchie at AT&T which uses a totally unified name space. Unix unified I/O and the file system to some extent with the /dev stuff, which let you treat a raw device as a file. It was also the first system to let you treat partitions and remote mounts as just subdirectories of a single name space. Plan 9 extends that to all OS entities with few limitations, including network connections, resources on remote machines, etc. As with the example above under Reiserfs, which should I have to write a completely different piece of code to get stuff off an FTP server vs off my local hard drive? Everything needs to be a file (or some more advanced successor to the concept of a file). This will wipe away much of the idiocy of current system design where 97 incarnations of basically the same code get written and rewritten every day.
I think we all know more or less what Transmeta is doing. The Code Morphing Software translates one machine language to another on the fly using custom silicon which means that it is VERY fast. I see no reason why it is in any way limited to x86 code. It could translate PPC code to the Carusoe instruction set just as easily. What about Java bytecode? It should be able to do a pretty good job with that. And I see no reason why multiple CMS implementations shouldn't coexist happily in theory, so that your kernel might be written in one machine code, and applications in other ones! This provides the necessary bridge to the future, where we may well see one single standard machine language, or at least generations which can smoothly migrate upwards, ending the chaos of hardware architectures.
XSLT is a W3C project which specifies how to create a XSL stylesheet which can be "applied" to an XML document to produce some arbitrary transformation. This sort of technology lets us treat programs themselves as data, since XSL is itself XML. Technically any "Turing Complete" language is logically complete and can transform its own code in arbitrary ways, but XSLT really brings these capabilities down to a useable level. When combined with a better concept for what a "file" is, we can do some very powerful stuff!
3. The system of the future
I see the data processing systems of the future as representing essentially a "seamless compute fabric". To a user the distinction between systems will vanish and only the interface will be apparent. 50 years from now it will be senseless to ask what sort of computer, or what process is performing any certain task. There will be what appears to be a single all pervasive computing infrastructure of massive power unimagined in even the wildest Sci Fi of today. Ordinary people will dispose of PetaFLOPS of compute power.
From a system perspective what will the architecture be like? Obviously with such huge resources we will be able to attack VERY hard problems by sheer brute force. The range of usefull functions for systems will be so huge that we will no longer be able to afford to program them at the statement level we use today. We see this in "Star Trek" whenever someone says "Computer, show me...". At the least we need formal grammers that are very powerful and allow us to specify operations at a much higher level than we do now. Something like "Merge the two user databases at X and Y, filtering out all record not present in Z and store the result to Q" would be a start. Incompatible schemas would be dealt with, fetching the data would be invisible to code. None of the gory details we deal with today are going to be visible to programmers of 50 years from now.
Specifically I see XML/XSL/XSLT and the SAX and DOM API's and their successor technologies as giving us a "lingua franca" for data representation and manipulation which will consume all other representations in the long run. Something like Reiserfs could be a very efficient XML database, storing each property and value in seperately named places, and providing direct SAX and DOM interfaces to the system. A universal naming system like Plan 9's would extend that functionality to the whole network and make it look like a fairly seamless single database. With CMS-like applications providing complete hardware transparency such a system would provide the logical platform for this seamless compute fabric.
Naturally we would use existing technologies like IP, MIME, HTTP, CORBA, etc as underlying elements (or their successors). There would also be management pieces, but those present no real fundamental hurdles. We will simply harvest the system's vast resources to let it manage itself to a large extent, reconfiguring and rerouting as necessary to fix problems.