In a nutshell all of the components of the OS, namely the Kernel, system applications, and whatever bloatware the carrier decided to add, are what make up the ROM.
The Kernel itself includes the core elements of the operating system. This would likely include components such as memory management, task scheduling, the file system, accessing the various hardware elements of the phone.