Once I had the status-driven SCSI driver, it was easy to write the tape and disk driver to use it. Soon, the tape version of mkfs and restore could create a file system on the disk.
The rest of Adaptive’s STM/18 project bustled along as well. Boards had begun to appear on the hardware folks’ tables, all with square plastic packets fringed with silver leads that shimmered against the deep green substrate.
It was time to tackle the kernel itself. Few people have had the pleasure of working on a Unix system boot from the release of the RESET signal to the first shell prompt, which is really too bad because it’s so much fun.
On power-up, the circuitry on the board keeps a RESET signal asserted for a set amount of time, which forces the chips to initialize into a known state. After this delay, the RESET releases and the CPU starts up.
It looks in the ROM, and reads a program counter from location zero and a stack pointer from location four. This begins the bootloader’s execution.
Also in ROM is a small, status-driven SCSI driver that reads in the first 512-byte sector from SCSI disk zero. This one sector is enough to load the file /boot from the root file system. This much larger 5K bootstrap stage knows how to load the full kernel, known as /unix. (Actually, /boot will prompt for which kernel you want to boot, which is handy when working on new kernels.)
The Unix kernel is just a binary, a normal compile like any other in the system. I tell the loader to run at 0x80000000, or halfway into the 32-bit addressable at the beginning of the third gigabyte. This is where Unix has loaded since the DEC VAX in the ‘70s. The MIPS processor also built this into the architecture. For the Motorola 68030, it was just a convention, not part of how the CPU worked.
The kernel, which is just a normally compiled program, is loaded in the beginning of physical memory. When I talk about physical memory, I’m referring to the way memory looks to the memory chips. My ’147 board had 4 MB, and the first byte of physical memory wasn’t necessarily going to be the first byte in a running program.
The illusion is done by the MMU, that part of the processor that takes the program addresses, what we call “virtual,” and turns them into “physical” addresses.
One way to think about memory is to visualize a tall rectangle. Memory begins at the bottom of the rectangle with the address of zero. The top is 0xffffffff, the highest address we can have in a 32-bit system.
With this rectangle in mind, imagine the physical address space as well as the virtual one. The design of the hardware defines what the physical address space looks like. For the MVME147, the lower part of the rectangle contained the dynamic RAM, the stuff that typically comes to mind when we think of memory. On my board for the Unix port this went from 0x00000000 to 0x00400000 for a total of 4MB. Later boards went up to 32MB.
From the end of local RAM to 0xEFFFFFFF, the 3 GB limit, would be the memory space for VME bus devices. Later, we actually added more memory to the system by just inserting a VME card that was nothing but memory. Worked right out of the box.
At the top of the memory rectangle are some areas for I/O chips. Things like Ethernet and SCSI controllers, timers, and so on are found in these high addresses of physical memory.
So that’s the visual view of the physical memory rectangle; the virtual view looks entirely different. In Unix, each program thinks it’s running all by itself in memory and is unable to “see” any other running programs. Obviously they each have to occupy different addresses, so our physical rectangle would show them at different physical locations.
In the virtual rectangle, user programs and instructions are in a segment starting at the bottom. In Unix parlance this is called the “text” segment. On top of that there is the global data for our program. Just north of this data is a kind of “no man’s land.” Attempts by a program to read or write in this area results in a crash and a dropped core file.
Halfway up our rectangle is the stack segment. Every time a new function is called, a stack frame is created to hold the variable local to the C function. In this way, a function can call itself and get a new set of local variables. If the program touches memory right below the currently allocated stack, the kernel will see a page fault but is clever enough to allocate you more stack and change the MMU to map that into your virtual rectangle.
The top of our virtual rectangle contains the same kind of no man’s land as the space below the stack segment. Until the process switches into kernel mode, that is.
Since the early days of timesharing, hardware has had two ways to operate: user and kernel mode. The MMU knows which mode you’re in.
When a user process issues a system call, it’s in effect just calling a strange library from kernel code. Since the VAX days, Unix maps the kernel into the virtual address space in the upper half of our virtual rectangle. When the user program wants to do something that’s in the kernel, it loads a value into a register and issues some sort of trap instruction.
The trap is caught by the kernel, just like a page fault, and the kernel knows what the user process is asking for. When it goes into kernel mode, kernel addresses starting at 0x80000000 magically appear. The kernel can access the user program in the lower part of the rectangle, and all of its addresses as well. As its addresses were still being translated, the kernel was loaded into address 0x80000000 in physical memory. In fact, it’s loaded at location 0! But the virtual to physical translation of the MMU makes it appear to start halfway up the virtual address space.
When done, the trap returns and the user mode once again hides the kernel space from the user program.
It was getting late in the year, and more and more of this was working. By this point, I had the kernel working enough to move development off my Sun and onto the VME system. I was finally able to use the system to develop itself.
I used a VT100 as a console and UUCP to copy programs back and forth from the Sun to the new Unix system. This only required a second serial port to log into the Sun and transfer the files.
Over Thanksgiving weekend I worked from home, and my productivity skyrocketed. By the following Monday, I realized how distracting the large room of cubicles at Adaptive could be. Frustrated, I packed the VME bucket and all its trimmings into a cardboard box and hauled it to my car. I needed the Sun too, so I rolled it through the parking lot on my office chair and headed home.
What’s the protocol at Adaptive for taking office equipment home? No idea. I didn’t ask. I just did it.
At home I set up a tiny lab in the second bedroom. I can’t remember what I used for a desk, but it must have been creative because Betsy and I didn’t have any furniture to speak of. I probably laid a door over two short filing cabinets. It wouldn’t have been the first time.
Soon, I was back at it and going fast. Working at home meant I could occasionally walk down to Pizza-A-Go-Go for a slice at lunchtime. It was there that I started to ponder networking. But that’s for next week’s post.
Inventor, coder, and entrepreneur, Brantley Coile invented Stateful packet inspection, network address translation, and Web load balancing used in the Cisco LocalDirector. He went on to create the Coraid line of storage appliances, a product he continues to improve today.