Jerry Jelinek's blog

Search
Close this search box.

Month: March 2005

I am one of the engineers working on Solaris Volume manager. There are some questions I get
asked a lot and I thought it would be useful to get this information out to a broader
audience so I am starting this blog. We have also done a lot of work on the volume manager
in the recent Solaris
releases and I’d like to talk about some of that too.

One of the questions I get asked most frequently is how to do root mirroring on x86
systems. Unfortunately our docs don’t explain this very well yet. This is a short
description I wrote about root mirroring on x86 which hopefully explains how to set
this up.

Root mirroring on x86 is more complex than is root mirroring
on SPARC. Specifically, there are issues with being able to boot
from the secondary side of the mirror when the primary side fails.
Compared to SPARC systems, with x86 machines you have be sure
to properly configure the system BIOS and set up your fdisk partitioning.

The x86 BIOS is analogous to the PROM interpreter on SPARC.
The BIOS is responsible for finding the right device to boot
from, then loading and executing the master boot record from that
device.

All modern x86 BIOSes are configurable to some degree but the
discussion of how to configure them is beyond the scope of this
post. In general you can usually select the order of devices that
you want the BIOS to probe (e.g. floppy, IDE disk, SCSI disk, network)
but you may be limited in configuring at a more granular level.
For example, it may not be possible to configure the BIOS to boot from
the first and second IDE disks. These limitations may be a factor
with some hardware configurations (e.g. a system with two IDE
disks that are root mirrored). You will need to understand
the capabilities of the BIOS that is on your hardware. If your
primary boot disk fails and your BIOS is set up properly, then it
will automatically boot from the second disk in the mirror.
Otherwise, you may need to break into the BIOS while the machine is
booting and reconfigure to boot from the second disk or you may even need
to boot from a floppy with the Solaris Device Configuration Assistant (DCA)
on it so that you can select the alternate disk to boot from.

On x86 machines fdisk partitions are used and it is common to have
multiple operating systems installed. Also, there are different flavors
of master boot programs (e.g. LILO or Grub), in addition to the standard
Solaris master boot program. The boot(1M) man page is a good resource
for a detailed discussion of the multiple components that are used during
booting on Solaris x86.

Since SVM can only mirror Solaris slices within the Solaris fdisk
partition this post focuses on a configuration that only
has Solaris installed. If you have multiple fdisk partitions
then you will need to use some other approach to protect the data
outside of the Solaris fdisk partition. SVM can’t mirror that data.

For an x86 system with Solaris installed there are two common
fdisk partitioning schemes. One approach uses two fdisk partitions.
There is a Solaris fdisk partition and another, small fdisk
partition of about 10MB called the x86 boot partition.
This partition has an Id value of 190. The Solaris system installation
software will create a configuration with these two fdisk partitions
as the default. The x86 boot partition is needed in some cases,
such as when you want to use live-upgrade on a single disk configuration,
but it is problematic when using root mirroring. The Solaris
system installation software only allows one x86 boot partition for
the entire system and it places important data on that fdisk
partition. That partition is mounted in the vfstab with this
entry:

/dev/dsk/c2t1d0p0:boot – /boot pcfs – no –

Because this fdisk partition is outside of the Solaris fdisk
partition it cannot be mirrored by SVM. Furthermore, because
there is only a single copy of this fdisk partition it represents
a single point of failure.

Since the x86 boot partition is not required in most cases
it is recommended that you do not use this as the default when setting
up a system that will have root mirroring. Instead, just use
a single Solaris fdisk partition and omit the x86 boot partition for
your installation. It is easiest to do this at the time you install
Solaris. If you already have Solaris installed and you created
the x86 boot partition as part of that process then the easiest
thing would be to delete that with the fdisk(1M) command and reinstall,
taking care not to create the x86 boot partition during the installation
process.

Once your system is installed you create your metadbs and root
mirror using the normal procedures.

When you use fdisk to partition your second disk you must make the
disk bootable with a master boot program. The -b option of fdisk(1M)
does this.

e.g.
fdisk -b /usr/lib/fs/ufs/mboot /dev/rdsk/c2t1d0p0

On x86 machines the Solaris VTOC (the slices within the
Solaris fdisk partition) is slightly different from what is seen
on SPARC. On SPARC there are 8 VTOC slices (0-7) but on x86
there are more. In particular slice 8 is used as a “boot”
slice. You will see that this slice is 1 cylinder in size
and starts at the beginning of the disk (offset 0). The other
slices will come after that, starting at cylinder 1.

Slice 8 is necessary for booting Solaris from this fdisk partition. It
holds the partition boot record (pboot), the Solaris VTOC for the disk
and the bootblk. This information is disk specific so it
is not mirrored with SVM. However, you must ensure that
both disks are bootable so that you can boot from the secondary
disk if the primary fails. You can use the installboot program
to setup the second disk as a Solaris bootable disk (see installboot(1M)).
An example command is:

installboot /usr/platform/i86pc/lib/fs/ufs/pboot \
/usr/platform/i86pc/lib/fs/ufs/bootblk /dev/rdsk/c2t1d0s2

There is one further consideration for booting an x86 disk. You must
ensure that the root slice has a slice tag of “root” and the root slice
must be slice 0. See format(1M) for checking and setting the slice tag
field.

Solaris x86 emulates some of the behavior of the SPARC eeprom. See
eeprom(1M). The boot device is stored in the “bootpath” property that
you can see with the eeprom command. The value is the device tree path
of the primary boot disk. For example:

bootpath=/pci@0,0/pci8086,2545@3/pci8086,1460@1d/pci8086,341a@7,1/sd@0,0:a

You should set up the alternate bootpath via the eeprom command so that
the system will try to boot from the second side of the root mirror.
First you must get the device tree path for the other disk in the mirror.
You can use the ls command. For example:

# ls -l /dev/dsk/c2t1d0s0 lrwxrwxrwx 1 root root 78 Sep 28 23:41 /dev/dsk/c0t1d0s0 -> ../../devices/pci@0,0/pci8086,2545@3/pci8086,1460@1d/pci8086,341a@7,1/sd@1,0:a

The device tree path is the portion of the output following “../devices”.

Use the eeprom command to set up the alternate boot path. For example:

eeprom altbootpath=’/pci@0,0/pci8086,2545@3/pci8086,1460@1d/pci8086,341a@7,1/sd@1,0:a’

If your primary disk fails you will boot from the secondary disk.
This may either be automatic if your BIOS is configured properly,
you may need to manually enter the BIOS and boot from the secondary
disk or you may even need to boot from a floppy with the DCA on it.
Once the system starts to boot it will try to boot from the “bootpath”
device. Assuming the primary boot disk is the dead disk in the root
mirror the system will attempt to boot from the “altbootpath” device.

If the system fails to boot from the altbootpath device for some reason then
you need to finish booting manually. The boot should drop in to the Device
Configuration Assistant (DCA). You must choose the secondary disk as the
boot disk within the DCA. After the system has booted you should update the
“bootpath” value with the device path that you used for the for the secondary
disk (the “altbootpath”) so that the machine will boot automatically.

For example, run the following to set the boot device to the second
scsi disk (target 1):

eeprom bootpath=’/pci@0,0/pci8086,2545@3/pci8086,1460@1d/pci8086,341a@7,1/sd@1,0:a’

Note that all of the usual considerations of mddb quorum apply
to x86, just as they do for SPARC.

Recent Posts

September 23, 2010
September 13, 2010
May 26, 2009

Archives