Jerry Jelinek's blog

Search
Close this search box.

Month: February 2006

Although I haven’t been working on SVM for over 6 months (I am
working on Zones now), I still get
questions about SVM and x86 root mirroring from time to time. Some
of these procedures are different when using the new x86 boot
loader (GRUB) that is now part of Nevada and S10u1. I have some old
notes that I wrote up about 9 months ago that describe the
updated procedures and I think these are still valid.

Root mirroring on x86 is more complex than is root mirroring
on SPARC. Specifically, there are issues with being able to boot
from the secondary side of the mirror when the primary side fails.
On x86 machines the system BIOS and fdisk partitioning are the
complicating factors.

The x86 BIOS is analogous to the PROM interpreter on SPARC.
The BIOS is responsible for finding the right device to boot
from, then loading and executing GRUB from that device.

All modern x86 BIOSes are configurable to some degree but the
discussion of how to configure them is beyond the scope of this
document. In general you can usually select the order of devices that
you want the BIOS to probe (e.g. floppy, IDE disk, SCSI disk, network)
but you may be limited in configuring at a more granular level.
For example, it may not be possible to configure the BIOS to probe
the first and second IDE disks. These limitations may be a factor
with some hardware configurations (e.g. a system with two IDE
disks that are root mirrored). You will need to understand
the capabilities of the BIOS that is on your hardware. If your
primary boot disk fails you may need to break into the BIOS
while the machine is booting and reconfigure to boot from the
second disk.

On x86 machines fdisk partitions are used and it is common to have
multiple operating systems installed. Also, there are different flavors
of master boot programs (e.g. LILO) in addition to GRUB which is the standard
Solaris master boot program. The boot(1M) man page is a good resource
for a detailed discussion of the multiple components that are used during
booting on Solaris x86.

Since SVM can only mirror Solaris slices within the Solaris fdisk
partition this discussion will focus on a configuration that only
has Solaris installed. If you have multiple fdisk partitions
then you will need to use some other approach to protect the data
outside of the Solaris fdisk partition.

Once your system is installed you create your metadbs and root
mirror using the normal procedures.

You must ensure that both disks are bootable so that you can boot from
the secondary disk if the primary fails. You use the installgrub program
to setup the second disk as a Solaris bootable disk (see installgrub(1M)).
An example command is:

/sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t1d0s0

Solaris x86 emulates some of the behavior of the SPARC eeprom. See
eeprom(1M). The boot device is stored in the “bootpath” property that
you can see with the eeprom command. The value should be assigned to the
the device tree path of the root mirror. For example:

bootpath=/pseudo/md@0:0,10,blk

Next you need to modify the GRUB boot menu so that you can manually boot
from the second side of the mirror, should this ever be necessary.
Here is a quick overview of the GRUB disk naming convention.

(hd0),(hd1) — first & second BIOS disk (entire disk)

(hd0,0),(hd0,1) — first & second fdisk partition of first BIOS disk

(hd0,0,a),(hd0,0,b) — Solaris/BSD slice 0 and 1 on first fdisk
partition on the first BIOS disk

Hard disk names starts with hd and a number, where 0 maps to BIOS
disk 0x80 (first disk enumerated by the BIOS), 1 maps to 0x81, and so on.
One annoying aspect of BIOS disk numbering is that the order may change
depending on the BIOS configuration. Hence, the GRUB menu may become
invalid if you change the BIOS boot disk order or modify the disk
configuration. Knowing the disk naming convention is essential to
handling boot issues related to disk renumbering in the BIOS.
This will be a factor if the primary disk in the mirror is not seen by
the BIOS so that it renumbers and boots from the secondary disk in the
mirror. Normally this renumbering will mean that the system can
still automatically boot from the second disk, since you configured
it to boot in the previous steps, but it becomes a factor
when the first disk becomes available again, as described below.

You should edit the GRUB boot menu in /boot/grub/menu.lst and add
an entry for the second disk in the mirror.
It is important that you be able to manually boot from
the second side of the mirror due to the BIOS renumbering described
above. If the primary disk is unavailable, the boot archive on that
disk may become stale. Later, if you boot and that disk is available
again, the BIOS renumbering would cause GRUB to load that stale boot
archive which could cause problems or may even leave the system unbootable.

If the primary disk is once again made available and then you reboot without
first resyncing the mirror back onto the primary drive, then you
should use the GRUB menu entry for the second disk to manually boot from
the correct boot archive (the one on the secondary side of the mirror).
Once the system is booted, perform normal metadevice maintenance to resync
the primary disk. This will restore the current boot archive to the
primary so that subsequent boots from that disk will work correctly.

The previous procedure is not normally necessary since you would replace the
failed primary disk using cfgadm(1M) and resync but it will be required
if the primary is simply not powered on, causing the BIOS to miss the disk
and renumber. Subsequently powering up this disk and rebooting would
cause the BIOS to renumber again and by default you would boot from the
stale disk.

Note that all of the usual considerations of mddb quorum apply
to x86 root mirroring, just as they do for SPARC.

Recent Posts

September 23, 2010
September 13, 2010
May 26, 2009

Archives