In Xen, Dom0 is the main OS that handles all the Xen virtual domains (called domU).
As of version 4.2, Xen can be built natively on NetBSD without patches, although I haven't had success getting it working – the libraries don't link correctly.
First, do a NetBSD install as normal. This guide uses NetBSD amd64 6.0.1, and builds everything from source using pkgsrc and not the pre-built binaries.
There are two parts to the kernel for booting NetBSD with support for Xen as dom0. The first one is built by your system, and it is the kernel that NetBSD boots first. The second is a NetBSD GENERIC kernel provided by upstream, with Xen support. Both parts are needed to completely boot into a Xen environment where NetBSD can act as the primary domain.
Build and install the first Xen kernel:
cd /usr/pkgsrc/sysutils/xenkernel42 make install
Once it's built, there will be a xen.gz and a xen-debug.gz file in /usr/pkgsrc/sysutils/xenkernel42.
Copy the xen.gz kernel to the root directory, and unzip it.
The NetBSD Xen guide keeps the kernel as a gzipped file, but when using GRUB as a bootloader, it wouldn't work for me. Unzipping it works fine.
cp /usr/pkgsrc/sysutils/xenkernel42/work/xen.gz / gunzip /xen.gz
Next, download the Xen DOM0 kernel from NetBSD's FTP repository. This is a GENERIC kernel that ships with the OS, but rebuilt for Xen support.
ftp ftp://ftp.netbsd.org/pub/NetBSD/`uname -r`/`uname -p`/binary/kernel/netbsd-XEN3_DOM0.gz
Move it to the root filesystem and unpack it as well:
mv netbsd-XEN3_DOM0.gz / gunzip /netbsd-XEN3_DOM0.gz
As a master domain, you'll need some Xen tools installed to maintain installations.
cd /usr/pkgsrc/sysutils/xentools42 make install
There are two ways to boot into the NetBSD Xen kernel: using the original bootloader, or installing GRUB. I prefer using GRUB because I'm familiar with the syntax, and it can chainload into the NetBSD bootloader as well.
Install GRUB on the system:
cd /usr/pkgsrc/sysutils/grub make install
Create a configuration file for GRUB at /grub/menu.lst. This one boots into NetBSD Xen by default, using 512 MB of RAM. The bootloader is at the first partition of the first drive.
default 0 timeout 10 # Load NetBSD Xen through GRUB title NetBSD Xen root (hd0,0,a) kernel /xen dom0_mem=512M module /netbsd-XEN3_DOM0 bootdev=sd0a ro console=tty0 # Jump to NetBSD bootloader title NetBSD Bootloader root (hd0,0,a) chainloader +1
Install GRUB as your bootloader:
grub-install '(hd0)'
You can pass the device name directly as well, if you know it:
grub-install /dev/wd0d
An alternative way to install:
grub --no-curses root (hd0,0,a) setup (hd0)
You can alternatively use the default NetBSD bootloader, although I haven't gotten it to display output to the VGA console yet.
Add this menu entry to /boot.cfg:
menu=NetBSD Xen:load /netbsd.xen;multiboot /xen dom0_mem=512M
There are some example files installed with the xentools package that need to be copied over to the right locations so they can be used.
cp /usr/pkg/share/examples/rc.d/xen* /etc/rc.d/
Update /etc/rc.conf to start Xen on boot:
xenwatchdog=YES xencommons=YES xend=YES xendomains=YES
cp /usr/pkg/share/examples/xen/scripts/* /usr/pkg/etc/xen/
Setup the new Xen device files:
cd /dev && sh MAKEDEV xen
Now to create a NetBSD virtual machine.
For the base directories in this guide, I'm going to use /var/xen as the parent directory of all files. For the specific vm, /var/xen/netbsd
Bridge the main Ethernet device (bnx0 in this example) to create bridge0.
Create this file: /etc/ifconfig.bridge0
create !brconfig $int add bnx0 up
Reboot the dom0 to create the new Ethernet bridge.
Create a hard drive with as many GBs as you want. In this example, I'm using 4 GB (4096):
dd if=/dev/zero of=netbsd.img bs=1024k count=4096
Mount the drive, and give it an MBR:
vnconfig vnd0 netbsd.img fdisk -l /dev/vnd0d vnconfig -u vnd0
Create a new file, netbsd-xen:
# use this kernel to boot and install kernel = "/var/xen/netbsd-xen/netbsd-INSTALL_XEN3_DOMU" # use this kernel once install is complete # kernel = "/var/xen/netbsd-xen/netbsd-INSTALL_XEN3_DOMU" memory = 1024 name = "netbsd-xen" vcpus = 2 vif = [ 'mac=aa:00:00:50:02:f0, bridge=bridge0' ] disk = [ 'file:/var/xen/netbsd-xen/netbsd.img,0x1,w' ] root = "xbd0"
Install NetBSD as usual, using FTP to download the base install.
xm create -c netbsd-xen
Once you're finished, shutdown the install, and close it out with xm as well:
xm shutdown netbsd-xen xm destroy netbsd-xen
Edit the netbsd-xen config file, and switch the kernel to the GENERIC kernel with Xen support instead of the install one. Then, boot into NetBSD as normal!
xm create -c netbsd-xen
I haven't been able to get this working, but this is the basic setup:
They need GRUB installed on the root disk. Pygrub then runs from NetBSD and boots off the GRUB install of that box.
So, you need an image that is already created and formatted before doing anything. I haven't had much luck in trying to do this *from within NetBSD*. So for now, I'm going to go with using xen-create-image on another Linux box, and copying it over.
Here is a cobbled-together ubuntu-xen.cfg that probably won't work as-is:
bootloader = '/usr/pkg/bin/pygrub'
vcpus = '1'
memory = '512'
root = '/dev/vnd0 ro'
disk = [
'file:/var/xen/ubuntu-xen/swap.img,xvda1,w',
'file:/var/xen/ubuntu-xen/disk.img,0x300,w'
]
name = 'ubuntu-xen'
dhcp = 'dhcp'
vif = [ 'mac=00:16:3E:E8:0E:55, bridge=bridge0' ]
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
Download the Ubuntu Xen kernel, initrd, and Xen config files from the FTP repo.
Create a config file for Xen:
kernel = "/var/xen/ubuntu/vmlinuz" ramdisk = "/var/xen/ubuntu/initrd.gz" name = "ubuntu" memory = "2048" disk = [ 'file:/var/xen/ubuntu/disk.img,xvda,w', ] vcpus = 4 on_reboot = 'restart' on_poweroff = 'destroy' on_crash = 'destroy' vif = ['mac=00:16:3e:00:00:11, bridge=bridge0']
Boot the new Xen image and install Ubuntu:
xm create -c config