The Target Task
About two years ago, I had a desire to replay one of my old adventure games, a SegaSoft game called The Space Bar. I became distressed when it would not work on my newest machine. I became more distressed when I went to the SegaSoft site and found no mention of it whatsoever, even though it was only three years old. I was, eventually, able to hack it into working, but the experience pointed out a bigger problem, that of abandonware.
Many people have suggested that computer games are an art form. Others think the word "art" is too big and prefer to call them entertainment. Whatever one's view of the semantics, games are a major cultural phenomenon, already comparable in revenues to the film industry. It seems likely, or at least possible, that people will want to study the history of games a hundred years from now. Yet they are now only in their infancy. The major problems with early films were twofold: the use of explosive, rapidly aging cellulose stock, and the technical complexity of entering the field. Technological improvements paved the way for films to become at least a polished form of entertainment, and in some instances, genuine art. Technology isn't the solution to every problem in the world, but removing technological obstacles is often to the good. Analogous in the computer game field are abandonware proprietary systems and the difficulty of using the tools. I set out to ameliorate these barriers by developing a new game editor and environment with the working title of CAGEE. Phase I of this development addresses cinematic adventure games; Phase II will build on this core to address 3-D geometric games.
Many people have told me this is futile and pointless. I don't much care; at least I'm having fun. Anyway, I'm presenting this only as the backdrop for my experience with Cocoa.
Although I have worked on much larger software projects before, this one has presented certain challenges. I decided that all of the runtime environment should use free software exclusively that could be made to run on many platforms but which did not restrict in any way the copyright or licensing of the actual games produced. Technically, this is easily done, as most games have stripped-down user interfaces that are easy to duplicate under a minimal API assuming a keyboard, mouse, and screen. I also decided that the editors should, on the other hand, provide the kind of user interface that is expected on the target machine. The less a user is annoyed, the better work they do. At the same time, the editor needed to share nearly all of its code with the runtime environment to avoid incompatibility issues. It could not be a toy system, like WorldBuilder, but is required to provide performance and features comparable to any commercial games, which requires a fast language.
I decided to do simultaneous development in Mac Cocoa and vanilla C with Posix file access for a VT100 terminal or even a glass teletype. I figured these two environments were far enough apart that any future planned systems (X on various UN*X flavors, Win32 and Win64, Palm OS, and whatever else I can't think of) would have a good chance of lying somewhere between these two.
History of Cocoa
The basic ideas of Cocoa came from NextStep, a system developed for the NeXT computer system. Like the Xerox Alto, the NeXT contained innovative ideas on hardware that was not up to the task. Later, this spawned OpenStep, an open specification with a free GNU implementation. When Steve Jobs came back to Apple, he brought these ideas to the Mac. Cocoa is intended to be used for all new development on the Mac, replacing Carbon, the partial substitute for the old Macintosh Toolbox routines. Its NextStep history lives on in the fact that every class and type starts with "NS." OpenStep still exists but is somewhat behind Cocoa in refinement.
Cocoa is natively programmed in Objective C, although it can also be programmed in Java. Objective C is a curious little language. It is about as old as the beginnings of C++ and was reasonably well developed some years before it was clear that C++ was going to be a big success. Unlike C++, Objective C adds little to the C language syntactically, but what it does add causes Objective C programs to look very unlike C. Although built around C, Objective C is more closely related to the ideas of Smalltalk 80. Like Java, it is a fully dynamic language with internal class introspection, a feature essential to the Cocoa API.
As a result of its history with NextStep and the free software community, Cocoa, unlike X, Win16 and the Macintosh Toolbox, has been fairly mature since the first day it appeared on the Mac.
Overview of Cocoa
Cocoa is deceptively simple. It consists of the Foundation and the Application Kit. Each has on the order of two hundred object classes and a couple dozen protocols (somewhat analogous to interfaces in Java). Nevertheless, these objects provide quite a lot of compact power. For example, three classes provide all needed OpenGL functionality, two classes handle URLs and net connectivity, one class provides Undo capability, and one class provides a multiple-font, formatted text editor that supports a good chunk of Rich Text Format (RTF). The bigger classes tend to be somewhat broad in functionality. The NSString class contains methods for formatting file names, encoding and decoding UTF-8, and getting a string via a URL.
Cocoa is claimed to rely heavily on the "model/view/controller paradigm." (For those who are more familiar with traditional C++ design, this corresponds roughly to "business class/interface class/connections between them." In reality, the situation is more complex. Controllers are used to do many things other than communicate between user-interface objects and computational objects. For example, the NSDocumentController is responsible for creating the correct NSDocument objects with opened files based on file type or extension. In addition, there is a catch-all delegate role for many objects. It is not at all uncommon for a nominal controller or other object to function as the delegate for several objects in addition to its primary role. Perl may be the epitome of TMTOWTDI (There's More than One Way to Do It), but Cocoa comes fairly close. This can cause some confusion as to which is the "correct" way, that is, the way that minimizes the chances of painting oneself into a corner in the future. In Cocoa, the way that is easiest generally turns out to be the best.
This characterizes most of my experience with Cocoa. I am used to Java, Win32, Palm OS, X, the old Macintosh Toolbox, SGI, and sloppier systems. They have trained me to intuit that, while everything is fine when you do exactly the kind of tasks in the examples given for the API, once you start to vary even a little bit as you need to do a real application, it rapidly becomes more productive to rewrite parts from scratch. Cocoa almost never gives that experience. Almost always, there is a clean way of doing anything. As a result, Cocoa takes some getting used to for the seasoned veteran but thereafter is largely delightful. Cocoa has some inconsistencies and historical problems (e.g. NSString does not properly distinguish between Unicode and UTF-16), their number is remarkably small. Implementation problems are more common than design problems.
The Development Environment
The development environment consists of the Project Builder and the Interface Builder. These are freely downloadable from Apple and are also available on a disc for about $20. They come with plenty of examples and documentation. Some of the documentation is incomplete or contradictory, but there is enough to do effective Cocoa programming with little or no recourse to other materials.
Project Builder (PB) appears as a fairly standard IDE. It has projects with a hierarchical view of files, a class browser, global find, debugging buttons that look like a VCR, just what you'd expect. It is interesting, however, in two ways. One is that it is aware of the Interface Builder and the Cocoa application structure. The other is that it is basically a skin over a modified version of the GNU gcc compiler and debugger, albeit one that is so well integrated it is hard to tell the difference between the experience of using it and of using CodeWarrior or Visual Cafe. The debugging buttons and integrated breakpoints work fine, but it is always possible for the power user to drop into the debugger using text.
Before going on to the Interface Builder, a few words about Mac applications are needed. Mac applications traditionally used resource forks, which provided some information for the running application, including strings, window positions, etc. Resources are also used in the Windows world, though not for as many uses. The traditional Macintosh resource fork was an extension to the file system, only available on the Macintosh (though the Be OS later expanded on these ideas). With the new UN*X (Mach/BSD) underpinnings of OS X, this additional fork becomes more problematic. While OS X is normally installed with a file system that supports them, UN*X is UN*X, and when talking about files, that means Posix. The resource fork goes away. In its place is a thing called a bundle. A bundle is a UN*X subdirectory that behaves as a single file to the GUI (but not to the shell, making it easy to look inside them). This bundle contains a number of files, including the following:
- Executable code
- Libraries (serving a similar purpose to DLLs)
- Strings, images, and other, possibly organized by language
- Nib files, possibly organized by language
As anyone who has every built an object system completely from scratch knows, it's the 5% that doesn't drop straight out of the object paradigm that's the real bear. Every object system has a certain amount of FM (more politely, "magic") to deal with this 5%. Nib files hold most of the Cocoa FM. A nib file specifies a set of classes and instances used as dynamic templates by the application.
Interface Builder (IB) is the program that edits nib files. It appears as an ordinary visual window layout manager. I have a violent allergy to visual layout managers in general, finding them clunky and confining. This is probably the first I have ever seen that is worth using.
While PB and IB can build many types of applications and libraries, the most common is the document-based application, which permits multiple documents, each with one window (usually). These applications usually have at least two nib files: one for the main menu and one for each distinct kind of document. They may have other nib files for dialogue windows, etc., but these (and the main menu) are easy. The document nib files are more interesting.
Document nib files typically contain at least three objects:
- A window
- The file's owner
- The first responder
The window is straightforward; it can be edited and filled with views in a manner familiar from many other systems. Users of Swing or even the older Java AWT will recognize how views are laid out within windows as a simpler analogue to those systems. The file's owner is somewhat flexible. It is usually an instance of the document class but may be a controller or other object as well. It is called the file's owner because the code for this object in the project is responsible for loading or at least specifying the name of the nib file to load. The first responder, however, is pure FM. It is a placeholder for any object that is found at run time by traversing the "responder chain." This includes any object that might possibly be interested in a user interface action, such as a button press or text box change.
Objects are linked to each other two ways: through outlets and actions. When a document is loaded, the run-time system consults the nib and creates instances specified within the nib.
Outlets and actions are identified within code using the
IBOutlet type modifier for outlets and
IBAction for actions. These are typedefed to innocuous values at compile time but still alert the compiler to produce some of the FM for the run time. Beyond this, there is no hairy extra syntax that needs to be maintained in the source to have it communicate with the nib files.
This raises the question of how the Project Builder and Interface Builder keep in sync. There are two ways, both essentially manual.
One can use the Interface Builder to create files for classes that have been defined within it. This is usually the best option when starting a new nib. It provides partial source files that can be expanded into functioning code. However, if you change your mind and add a new outlet or nib later, this will ask you if you want to merge the new files with the existing files. If you say so, it will bring up Merge, a manual merging tool that looks like an inferior version of SGI's gdiff. It is inferior both because the user interface is highly confusing (arrows point to the text to get from, in the style of weather diagrams, rather than showing where text will go) and because it has a funny idea of how to find similarities. Merge is somewhat usable, especially with the option to include both, but it is dangerous if you aren't careful.
A better way is to type the new outlets or connections into the header file and have IB read the file. It then sets up its internal information correctly. One caveat is that the class and subclass structure in Interface Builder must be the same as and use the same names as the code. Otherwise, IB will complain.
It is easy to subsitute custom subclasses for existing objects. An NSView, for instance, can be of any "custom class" that makes sense, including all of the programmer's subclasses of NSView.
Although this system looks like it would be fine for simple, classical applications, the question is whether it can be used to produce a real application? The answer appears to be "yes."
Due to the need for platform independence, I elected to specify a narrow channel of glue to go between the computational part and the user interface. The computational part is written in vanilla ANSI C, implementing a full LISP mark-and-sweep garbage-collection memory system. Nothing in the LISP portion is specific to Cocoa. (At first, the LISP strings were allowed to use either UN*X malloced arrays or NSStrings based on conditional compilation. Fortunately, I soon realized what a bad idea that was. Now, they are UN*X malloced strings explicitly specified to be UTF-8 encodings, converted to and from NSString objects when necessary. Although this requires a minimal amount of extra overhead, it is more than made up for by not being tied to a myopic implementation of Unicode.) There are seven Objective C glue classes, most of them very simple. The most important is
GlueObject, which links a LISP object to a Cocoa object. Also important is
GlueBinding, which makes it easy to write a user interface element in Cocoa or another system to edit the binding of a variable within an object. Each glue has a pointer to the LISP structure, and there is a special LISP expression called a "reference" that can hold a pointer of the appropriate size for the architecture.
The differing memory management systems required some thought. LISP, of course, has a full mark-and-sweep garbage collector, much like Java. Objective C has only a scheme based on reference counts. Fortunately, there are very strict rules on the ownership of objects that, when rigorously followed, prevent memory leaks. Any object that creates another object using the alloc method is responsible for releasing it when done. There is also an autorelease mechanism that keeps an object around long enough for it to be used, which lets factory methods work. To keep the LISP system in sync with the Objective C memory, I decided that the LISP system should work with three rules:
- Retain the object it references from LISP in addition to any other object that retains it
- Only delete during mark-and-sweep if nothing outside the LISP system has retained the object
- If a reference should not be deleted, neither should the LISP object that refers to it
To maintain independence, how to query whether an object is owned elsewhere and how to delete it is specified in vanilla C in the glue code.
CAGEE is written both in vanilla C and in C++, the latter mostly for interfacing with external libraries such as Apache Xerces. This turned out to be easy in the Project Builder which handles vanilla C, Objective C, C++, and Objective C with C++. The last does not, of course, provide any magic to make C++ look like Objective C or vice versa; the models are too different for that to be practical. It does get the job done, though.
Because CAGEE is a project-based application, it is a little bit more complicated than the typical document-based application. The project document owns and refers to additional documents which may or may not be stored in files. Unlike PB, CAGEE does not simply refer to other files but keeps the LISP code of a subfile in memory, accessible to the project. First, I had to rewrite the functionality of the New menu item to a New Project... item that asked to save first rather than allowing an untitled document to be saved later. PB, of course, does that, but with a somewhat baroque assistant mechanism. I just used the standard file save dialog window, adding a single view to allow the user to specify whether a new subdirectory should be created for the new project. Close Save, and Save As... continued to work fine. For the subdocuments, I also had to modify New to a context-sensitive system that created a document of the right kind (I still didn't want to have an assistant, which I view primarily as an indictment that the user interface is more complex than it should be). Everything else worked fine. The standard Cocoa document methods caused a modified LISP print to produce XML on output, and Apache Xerces to read an XML file on input. To open double-clicked files automatically, I added an openDocumentForObject: fromDoc: method to the shared MasterDocumentController, a custom subclass of NSDocumentController.
Undo was a bit tricky. Using the NSUndoManager, it is very easy to do any kind of undo and redo, provided that everything is in Cocoa. Unfortunately, it was also necessary to synchronize with the LISP memory system. The NSUndoManager does not retain arguments passed to it in order to avoid making circular references. There is a method to delete all undoable and redoable actions involving a certain object, but this is an extremely dangerous cheat. Undo and redo are only mathematically justifiable when you can guarantee that no undoable action can ever possibly be affected by any other action. To get around this, I bit the bullet and maintained separate LISP undo and redo stacks, with the obvious synchronization and cleanup when a window closes. It is not particularly elegant, but it has the pleasant side-effect of providing most of the work of undo and redo in vanilla C.
Unlike previous Macintosh development systems and to a lesser extent Windows and Java systems, Cocoa is friendly about the command line. When debugging, any stdio or cin/cout activity uses a panel within the Project Builder. I have a LISP read/eval/print loop patiently waiting in a separate thread. It is handy for debugging and testing, such as forcing a garbage collection in a critical state. The input facilities go away if the application is double-clicked, however.
I hope this article has given an impression of what it's like to use Cocoa. Yes, it's not only possible to do a fairly complex application in Cocoa, but it's as close to being fun as any API I've seen in a long time.