So good, in fact, that I made a firewall this way a couple years ago. But I can't take any credit for the idea; it's a project called PicoBSD. PicoBSD's makefiles and scripts easily build your custom kernel, plus a multi-run binary that incorporates many basic utilities, which then get wrapped in a filesystem image ready for you to write to a floppy disk. Yes, a floppy disk. At runtime, PicoBSD sets up and runs from an MFS, so your floppy can be read-only.
The firewall I set up ran on a Pentium with 8MB of RAM, and was not exploitable, because it did not accept network connections from anybody. It just passed packets. If somebody had managed to break in, there weren't enough utilities (or space) to do any damage, and since the system booted from a floppy disk, I could have updated the system configuration easily.
The best part of it? PicoBSD is free, and already included with the FreeBSD source code.