25.6 Diskless Operation

Updated by Jean-François Dockès. Reorganized and enhanced by Alex Dupre.

A FreeBSD machine can boot over the network and operate without a local disk, using filesystems mounted from an NFS server. No system modification is necessary, beyond standard configuration files. Such a system is relatively easy to set up because all the necessary elements are readily available:

There are many ways to set up diskless workstations. Many elements are involved, and most can be customized to suit local taste. The following will describe variations on the setup of a complete system, emphasizing simplicity and compatibility with the standard FreeBSD startup scripts. The system described has the following characteristics:

Caution: As described, this system is insecure. It should live in a protected area of a network, and be untrusted by other hosts.

All the information in this section has been tested using FreeBSD releases 4.9-RELEASE and 5.2.1-RELEASE. The text is primarily structured for 4.X usage. Notes have been inserted where appropriate to indicate 5.X changes.

25.6.1 Background Information

Setting up diskless workstations is both relatively straightforward and prone to errors. These are sometimes difficult to diagnose for a number of reasons. For example:

In this context, having some knowledge of the background mechanisms involved is very useful to solve the problems that may arise.

Several operations need to be performed for a successful bootstrap:

See also diskless(8) manual page.

25.6.2 Setup Instructions

25.6.2.1 Configuration Using ISC DHCP

The ISC DHCP server can answer both BOOTP and DHCP requests.

As of release 4.9, ISC DHCP 3.0 is not part of the base system. You will first need to install the net/isc-dhcp3-server port or the corresponding package.

Once ISC DHCP is installed, it needs a configuration file to run, (normally named /usr/local/etc/dhcpd.conf). Here follows a commented example, where host margaux uses Etherboot and host corbieres uses PXE:

default-lease-time 600;
max-lease-time 7200;
authoritative;

option domain-name "example.com";
option domain-name-servers 192.168.4.1;
option routers 192.168.4.1;

subnet 192.168.4.0 netmask 255.255.255.0 {
  use-host-decl-names on; (1)
  option subnet-mask 255.255.255.0;
  option broadcast-address 192.168.4.255;

  host margaux {
    hardware ethernet 01:23:45:67:89:ab;
    fixed-address margaux.example.com;
    next-server 192.168.4.4; (2)
    filename "/data/misc/kernel.diskless"; (3)
    option root-path "192.168.4.4:/data/misc/diskless"; (4)
  }
  host corbieres {
    hardware ethernet 00:02:b3:27:62:df;
    fixed-address corbieres.example.com;
    next-server 192.168.4.4;
    filename "pxeboot";
    option root-path "192.168.4.4:/data/misc/diskless";
  }
}
         
(1)
This option tells dhcpd to send the value in the host declarations as the hostname for the diskless host. An alternate way would be to add an option host-name margaux inside the host declarations.
(2)
The next-server directive designates the TFTP or NFS server to use for loading loader or kernel file (the default is to use the same host as the DHCP server).
(3)
The filename directive defines the file that Etherboot or PXE will load for the next execution step. It must be specified according to the transfer method used. Etherboot can be compiled to use NFS or TFTP. The FreeBSD port configures NFS by default. PXE uses TFTP, which is why a relative filename is used here (this may depend on the TFTP server configuration, but would be fairly typical). Also, PXE loads pxeboot, not the kernel. There are other interesting possibilities, like loading pxeboot from a FreeBSD CD-ROM /boot directory (as pxeboot(8) can load a GENERIC kernel, this makes it possible to use PXE to boot from a remote CD-ROM).
(4)
The root-path option defines the path to the root filesystem, in usual NFS notation. When using PXE, it is possible to leave off the host's IP as long as you do not enable the kernel option BOOTP. The NFS server will then be the same as the TFTP one.

25.6.2.2 Configuration Using BOOTP

Here follows an equivalent bootpd configuration (reduced to one client). This would be found in /etc/bootptab.

Please note that Etherboot must be compiled with the non-default option NO_DHCP_SUPPORT in order to use BOOTP, and that PXE needs DHCP. The only obvious advantage of bootpd is that it exists in the base system.

.def100:\
  :hn:ht=1:sa=192.168.4.4:vm=rfc1048:\
  :sm=255.255.255.0:\
  :ds=192.168.4.1:\
  :gw=192.168.4.1:\
  :hd="/tftpboot":\
  :bf="/kernel.diskless":\
  :rp="192.168.4.4:/data/misc/diskless":

margaux:ha=0123456789ab:tc=.def100
         

25.6.2.3 Preparing a Boot Program with Etherboot

Etherboot's Web site contains extensive documentation mainly intended for Linux systems, but nonetheless containing useful information. The following will just outline how you would use Etherboot on a FreeBSD system.

You must first install the net/etherboot package or port.

You can change the Etherboot configuration (i.e. to use TFTP instead of NFS) by editing the Config file in the Etherboot source directory.

For our setup, we shall use a boot floppy. For other methods (PROM, or MS-DOS program), please refer to the Etherboot documentation.

To make a boot floppy, insert a floppy in the drive on the machine where you installed Etherboot, then change your current directory to the src directory in the Etherboot tree and type:

# gmake bin32/devicetype.fd0
   

devicetype depends on the type of the Ethernet card in the diskless workstation. Refer to the NIC file in the same directory to determine the right devicetype.

25.6.2.4 Booting with PXE

By default, the pxeboot(8) loader loads the kernel via NFS. It can be compiled to use TFTP instead by specifying the LOADER_TFTP_SUPPORT option in /etc/make.conf. See the comments in /etc/defaults/make.conf (or /usr/share/examples/etc/make.conf for 5.X systems) for instructions.

There are two other undocumented make.conf options which may be useful for setting up a serial console diskless machine: BOOT_PXELDR_PROBE_KEYBOARD, and BOOT_PXELDR_ALWAYS_SERIAL (the latter only exists on FreeBSD 5.X).

To use PXE when the machine starts, you will usually need to select the Boot from network option in your BIOS setup, or type a function key during the PC initialization.

25.6.2.5 Configuring the TFTP and NFS Servers

If you are using PXE or Etherboot configured to use TFTP, you need to enable tftpd on the file server:

  1. Create a directory from which tftpd will serve the files, e.g. /tftpboot.

  2. Add this line to your /etc/inetd.conf:

    tftp   dgram   udp wait    root    /usr/libexec/tftpd  tftpd -l -s /tftpboot
    

    Note: It appears that at least some PXE versions want the TCP version of TFTP. In this case, add a second line, replacing dgram udp with stream tcp.

  3. Tell inetd to reread its configuration file:

    # kill -HUP `cat /var/run/inetd.pid`
    

You can place the tftpboot directory anywhere on the server. Make sure that the location is set in both inetd.conf and dhcpd.conf.

In all cases, you also need to enable NFS and export the appropriate filesystem on the NFS server.

  1. Add this to /etc/rc.conf:

    nfs_server_enable="YES"
    
  2. Export the filesystem where the diskless root directory is located by adding the following to /etc/exports (adjust the volume mount point and replace margaux corbieres with the names of the diskless workstations):

    /data/misc -alldirs -ro margaux corbieres
    
  3. Tell mountd to reread its configuration file. If you actually needed to enable NFS in /etc/rc.conf at the first step, you probably want to reboot instead.

    # kill -HUP `cat /var/run/mountd.pid`
    

25.6.2.6 Building a Diskless Kernel

If using Etherboot, you need to create a kernel configuration file for the diskless client with the following options (in addition to the usual ones):

options     BOOTP          # Use BOOTP to obtain IP address/hostname
options     BOOTP_NFSROOT  # NFS mount root filesystem using BOOTP info
   

You may also want to use BOOTP_NFSV3, BOOT_COMPAT and BOOTP_WIRED_TO (refer to LINT in 4.X or NOTES on 5.X).

These option names are historical and slightly misleading as they actually enable indifferent use of DHCP and BOOTP inside the kernel (it is also possible to force strict BOOTP or DHCP use).

Build the kernel (see Chapter 8), and copy it to the place specified in dhcpd.conf.

Note: When using PXE, building a kernel with the above options is not strictly necessary (though suggested). Enabling them will cause more DHCP requests to be issued during kernel startup, with a small risk of inconsistency between the new values and those retrieved by pxeboot(8) in some special cases. The advantage of using them is that the host name will be set as a side effect. Otherwise you will need to set the host name by another method, for example in a client-specific rc.conf file.

Note: In order to be loadable with Etherboot, a 5.X kernel needs to have the device hints compiled in. You would typically set the following option in the configuration file (see the NOTES configuration comments file):

hints      "GENERIC.hints"

25.6.2.7 Preparing the Root Filesystem

You need to create a root filesystem for the diskless workstations, in the location listed as root-path in dhcpd.conf. The following sections describe two ways to do it.

25.6.2.7.1 Using the clone_root Script

This is the quickest way to create a root filesystem, but currently it is only supported on FreeBSD 4.X. This shell script is located at /usr/share/examples/diskless/clone_root and needs customization, at least to adjust the place where the filesystem will be created (the DEST variable).

Refer to the comments at the top of the script for instructions. They explain how the base filesystem is built, and how files may be selectively overridden by versions specific to diskless operation, to a subnetwork, or to an individual workstation. They also give examples for the diskless /etc/fstab and /etc/rc.conf files.

The README files in /usr/share/examples/diskless contain a lot of interesting background information, but, together with the other examples in the diskless directory, they actually document a configuration method which is distinct from the one used by clone_root and the system startup scripts in /etc, which is a little confusing. Use them for reference only, except if you prefer the method that they describe, in which case you will need customized rc scripts.

25.6.2.7.2 Using the Standard make world Procedure

This method can be applied to either FreeBSD 4.X or 5.X and will install a complete virgin system (not only the root filesystem) into DESTDIR. All you have to do is simply execute the following script:

#!/bin/sh
export DESTDIR=/data/misc/diskless
mkdir -p ${DESTDIR}
cd /usr/src; make world && make kernel
cd /usr/src/etc; make distribution

Once done, you may need to customize your /etc/rc.conf and /etc/fstab placed into DESTDIR according to your needs.

25.6.2.8 Configuring Swap

If needed, a swap file located on the server can be accessed via NFS. One of the methods commonly used to do this has been discontinued in release 5.X.

25.6.2.8.1 NFS Swap with FreeBSD 4.X

The swap file location and size can be specified with BOOTP/DHCP FreeBSD-specific options 128 and 129. Examples of configuration files for ISC DHCP 3.0 or bootpd follow:

  1. Add the following lines to dhcpd.conf:

    # Global section
    option swap-path code 128 = string;
    option swap-size code 129 = integer 32;
    
    host margaux {
      ... # Standard lines, see above
      option swap-path "192.168.4.4:/netswapvolume/netswap";
      option swap-size 64000;
    }
           
    

    swap-path is the path to a directory where swap files will be located. Each file will be named swap.client-ip.

    Older versions of dhcpd used a syntax of option option-128 "..., which is no longer supported.

    /etc/bootptab would use the following syntax instead:

    T128="192.168.4.4:/netswapvolume/netswap":T129=0000fa00
    

    Note: In /etc/bootptab, the swap size must be expressed in hexadecimal format.

  2. On the NFS swap file server, create the swap file(s):

    # mkdir /netswapvolume/netswap
    # cd /netswapvolume/netswap
    # dd if=/dev/zero bs=1024 count=64000 of=swap.192.168.4.6
    # chmod 0600 swap.192.168.4.6
               
    

    192.168.4.6 is the IP address for the diskless client.

  3. On the NFS swap file server, add the following line to /etc/exports:

    /netswapvolume  -maproot=0:10 -alldirs margaux corbieres
           
    

    Then tell mountd to reread the exports file, as above.

25.6.2.8.2 NFS Swap with FreeBSD 5.X

The kernel does not support enabling NFS swap at boot time. Swap must be enabled by the startup scripts, by mounting a writeable file system and creating and enabling a swap file. To create a swap file of appropriate size, you can do like this:

# dd if=/dev/zero of=/path/to/swapfile bs=1k count=1 oseek=100000

To enable it you have to add the following line to your rc.conf:

swapfile=/path/to/swapfile

25.6.2.9 Miscellaneous Issues

25.6.2.9.1 Running with a Read-only /usr

If the diskless workstation is configured to run X, you will have to adjust the XDM configuration file, which puts the error log on /usr by default.

25.6.2.9.2 Using a Non-FreeBSD Server

When the server for the root filesystem is not running FreeBSD, you will have to create the root filesystem on a FreeBSD machine, then copy it to its destination, using tar or cpio.

In this situation, there are sometimes problems with the special files in /dev, due to differing major/minor integer sizes. A solution to this problem is to export a directory from the non-FreeBSD server, mount this directory onto a FreeBSD machine, and run MAKEDEV on the FreeBSD machine to create the correct device entries (FreeBSD 5.0 and later use devfs(5) to allocate device nodes transparently for the user, running MAKEDEV on these versions is pointless).

This, and other documents, can be downloaded from ftp://ftp.FreeBSD.org/pub/FreeBSD/doc/.

For questions about FreeBSD, read the documentation before contacting <questions@FreeBSD.org>.
For questions about this documentation, e-mail <doc@FreeBSD.org>.

Hosting by: Hurra Communications Ltd.
Generated: 2007-01-26 17:58:43