I’ll be putting together a small, diskless, mixed-node Gentoo cluster this week and hopefully detailing my progress on here. I recently got some interesting ideas from a fellow dev, Christian Zoffoli, on the gentoo-cluster mailing list. He had a setup something like this:
What it should do:
- Power on.
- Start PXE — look for DHCP, TFTP servers.
- Download the kernel and a uclibc initrd.
- Mount an NFS shared image (one per cluster).
- Autoconfigure.
- Start a full Gentoo system.
Technical details:
- Create a diskless boot setup. Set up DHCP, TFTP and NFS servers. The DHCP server tells nodes their hostnames and where to go next to get a kernel. (Remember to use -H to dhcpcd on the client or else it ignores the hostname.) The TFTP server provides the kernel and initrd, and the NFS server provides a full root filesystem.
- Create a layer using an embedded environment so we can have a diskless node up and running quickly. ROOT support in Gentoo’s Portage should allow us to do this fairly easily.
- The linuxrc within the initrd does much of the real work.
- Mount /proc.
- Read the hostname from /proc/sys/kernel/hostname.
- Parse /proc/net/pnp to find the boot server.
- Download a more complex linuxrc (linuxrc.real) from a TFTP server, so there can be a simple, statically generated initrd.
- Execute linuxrc.real
- Mount the NFS root with a full Gentoo installation in a common path [COM_P1].
- Mount a tmpfs in another common path within the prior one [COM_P2].
- Generate resolv.conf, hostname, dnsdomainname, etc. in $COM_P2.
- Download SSH keys (common to all nodes) to $COM_P2.
- Mount a tmpfs in /tmp, /var/lock, /var/run, /var/lib/init.d and /var/log. The logger sends messages to a primary logger on the master.
- Link $COM_P2 to the real location (in $COM_P1).
- Link and download wathever you need.
- Switch roots to the NFS share using pivot_root.
- Execute /sbin/init in $COM_P1 to start the normal Gentoo initialization.
Striving for greatness 