Solaris: How to Clone LDOM

Solaris ZFS based LDOMs can be cloned either by command or even by script.  This greatly speeds up deployments in an environment where Solaris VMs need to be built quickly. Oracles (formerly Sun Microsystems) LDOM virtualization technology can be automated and greatly simplified when combined with the ZFS filesystem. LDOM's are Sun's spin on VMWare, or at least at best can be roughly compared to VMWare. One difference being that LDOM's assign physical hardware as compared to VMWare where time slices on resources are used in typical configurations. This comparison is based on typical implementations. The LDOM's are based on the CMT processor architectures found in the T4's and T2 type SPARC systems. I won't get into a big description of the technology. Suffice it to say, it is highly configurable. In this blog, I show how from a  simple script, guest LDOM's can be created or cloned very quickly with a fully bootable and running system with an independent kernel and hardware.

Setting up primary control and service domains
In any LDOM configuration, at a minimum a control and service domain is required. The configuration used in this example does not support dual service domains for virtual IO which provides the ultimate in availability.

Create virtual disk service

# ldm add-vds primary-vds0 primary

Create virtual console

#  ldm add-vcc port-range=5000-5100 primary-vcc0 primary

Add virtual switch

      # ldm add-vsw net-dev=e1000g0 primary-vsw0 primary

Create control domain and give it services.  T4-X's don't need the SSL chip settings :), if you aren't using a T4-x system, then set-mau is needed.

# ldm set-mau 0 primary
# ldm set-vcpu 4 primary
# ldm start-reconf primary
# ldm set-memory 4G primary

Adding and setting spconfig to initial

# ldm add-spconfig initial
# shutdown -i6 -g0 now

Configuring the network component enabling networking between control and service domain needs to be done from the console. Otherwise you will blow your knees out from under you. Then it gets awkward 

# ifconfig vsw0 plumb
# ifconfig e1000g0 down unplumb
# fconfig vsw0 147.28.18.28 netmask 0xffffff00 broadcast + up
# mv /etc/hostname.e1000g0 /etc/hostname.vsw0

Enable vntsd

# svcadm enable vntsd
# reboot

Setup ZFS based golden image LDOM
First setup the storage for the LDOM. As an example, lets create a 1 TB zpool (Waaaay better than VxVM & LVM). So I run the zpool create command and feed it the device nodes for the 2 multipathed HDS LUN's.

# zpool create LDOM_disk c0t60060E8006FF03000000FF0300001500d0 c0t60060E8006FF03000000FF0300001501d0

# zpool status LDOM_disk
pool: LDOM_disk
state: ONLINE
scan: none requested
config:

NAME                                     STATE     READ WRITE CKSUM
LDOM_disk                                ONLINE       0     0     0
c0t60060E8006FF03000000FF0300001500d0  ONLINE       0     0     0
c0t60060E8006FF03000000FF0300001501d0  ONLINE       0     0     0

errors: No known data errors

# zfs create -V 15g LDOM_disk/golden

Ok, we've created a golden zfs volume under the LDOM_disk pool. Dam that was easy. SVM or VxVM/LVM as sweet as it used to be, can't compare to ZFS for configuration simplicity. Ok, move on. Lets build an LDOM with our basic configuration. I have a fully loaded T4 with 256 CPU threads, and oceans of silicon for memory. So I'm being generous here. But really, this is just my golden image, so it's not really important what my resources are here, as it won't be used. So I'll give it just enough resources to install and configure a healthy Solaris 10 kernel.

# ldm add-domain golden
# ldm add-vcpu 2 golden
# ldm add-memory 8 G golden
# ldm add-vnet vnet1 primary-vsw0 golden

Add disk to virtual disk server

# ldm add-vdsdev /dev/zvol/dsk/LDOM_disk/golden vol0@primary-vds0

Add disk to golden LDOM

# ldm add-vdisk vdisk0 vol0@primary-vds0 golden

Set autoboot var

# ldm set-var auto-boot?=false golden

Map ISO for boot and installation

# ldm add-vdsdev /root/sol-10-u10-ga2-sparc-dvd.iso iso@primary-vds0
# ldm add-vdisk vdisk_iso iso@primary-vds0 golden
# ldm bind-domain golden
# ldm start-domain golden
# ldm start-domain golden
LDOM golden started

# ldm list-bindings golden

NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
golden          active     -t—-  5000    24    32G      4.2%  20s

UUID
8939999e-0d72-e842-a93e-ae86b57c5dc6

MAC
00:14:4f:fb:c5:4f

HOSTID
0x84fbc54f

CONTROL
failure-policy=ignore
extended-mapin-space=off
cpu-arch=native

DEPENDENCY
master=

CORE
CID    CPUSET
1      (8, 9, 10, 11, 12, 13, 14, 15)
2      (16, 17, 18, 19, 20, 21, 22, 23)
3      (24, 25, 26, 27, 28, 29, 30, 31)

VCPU
VID    PID    CID    UTIL STRAND
0      8      1      100%   100%
1      9      1      0.0%   100%
2      10     1      0.0%   100%
3      11     1      0.0%   100%
4      12     1      0.0%   100%
5      13     1      0.0%   100%
6      14     1      0.0%   100%
7      15     1      0.0%   100%
8      16     2      0.0%   100%
9      17     2      0.0%   100%
10     18     2      0.0%   100%
11     19     2      0.0%   100%
12     20     2      0.0%   100%
13     21     2      0.0%   100%
14     22     2      0.0%   100%
15     23     2      0.0%   100%
16     24     3      0.0%   100%
17     25     3      0.0%   100%
18     26     3      0.0%   100%
19     27     3      0.0%   100%
20     28     3      0.0%   100%
21     29     3      0.0%   100%
22     30     3      0.0%   100%
23     31     3      0.0%   100%

MEMORY
RA               PA               SIZE
0x40000000       0x140000000      32G

CONSTRAINT
threading=max-throughput

VARIABLES
auto-boot?=false

NETWORK
NAME             SERVICE                     ID   DEVICE     MAC               MODE   PVID VID
MTU   LINKPROP
vnet1            primary-vsw0@primary        0    network@0  00:14:4f:f9:44:b9        1
1500
PEER                        MAC               MODE   PVID VID                  MTU   LINKPROP
primary-vsw0@primary        00:14:4f:fb:ed:02        1                         1500

DISK
NAME             VOLUME                      TOUT ID   DEVICE  SERVER         MPGROUP
vdisk0           vol0@primary-vds0                0    disk@0  primary
vdisk_iso        iso@primary-vds0                 1    disk@1  primary

VCONS
NAME             SERVICE                     PORT
golden          primary-vcc0@primary        5000

Get yourself onto the console and boot off of the ISO and install the operating system.

# telnet localhost 5000
Trying 127.0.0.1…
Connected to localhost.
Escape character is '^]'.

Connecting to console "golden" in group "golden" ….
Press ~? for control options ..
~
{0} ok

{0} ok devalias
vdisk_iso                /virtual-devices@100/channel-devices@200/disk@1
vdisk0                   /virtual-devices@100/channel-devices@200/disk@0
vnet1                    /virtual-devices@100/channel-devices@200/network@0
net                      /virtual-devices@100/channel-devices@200/network@0
disk                     /virtual-devices@100/channel-devices@200/disk@0
virtual-console          /virtual-devices/console@1
name                     aliases

The vdisk_iso is a device alias that was created in the step above. So you can boot and install off of this device. Use the OBP boot command as shown below.

{0} ok boot vdisk_iso:f

NOTE: jumpstart works great here too. But in this case, I had an iso handy.
Go through install process. Add whatever software you need, DNS settings. Whatever you need for your base configs. I typically just give the LDOM a bogus IP address because I will be changing in once I boot the new guest LDOM created from this golden image. You can also in some cases run a sys_unconfig command and start with a clean slate. It's your choice.
Stop and unbind the guest golden image domain.

# ldm stop golden

Snap shot the disk image.

# zfs snapshot LDOM_disk/golden@golden-image

We have a fully bootable LDOM image with whatever version of the operating system installed. This is perfect. We now use this image to create the future guest LDOM's by cloning the image.

Script to create ZFS based golden image LDOM
Use the following script based on building the LDOMs with the golden image. This of course can be tweaked to change features such as vcup's and memory. The script is basic with no error checking. It uses only 2 positional paramater, as in 2 arguments. The first arg $1 is the LDOM name, second $2 is the volume name for the storage.

# cat ./clone-LDOM.sh

#---------------------- START OF SCRIPT -----------------------

echo Setting up clone for $1 with $2 storage

sleep 2

zfs clone LDOM_disk/golden@golden-image LDOM_disk/$1

echo creating domain

ldm add-domain $1

ldm add-vcpu 2 $1

ldm add-memory 4G $1

echo network

ldm add-vnet vnet1 primary-vsw0 $1

echo storage

ldm add-vdsdev /dev/zvol/dsk/LDOM_disk/$1 ${2}@primary-vds0

echo adding disk to $1 LDOM

ldm add-vdisk vdisk0 ${2}@primary-vds0 $1

echo set autoboot var

ldm set-var auto-boot?=false $1

echo binding

ldm bind-domain $1

ldm start-domain $1

ldm list-domain $1

#---------------------- START OF SCRIPT -----------------------

Assumption is you have many vcpu's and tons of disk. Output of running the script. I called the script clone-LDOM.sh.  Lets run the script and create an LDOM and associate a ZFS volume called test-ldm-vol to the LDOM. Kay, lets go:

# ./clone-LDOM.sh test-ldom test-ldm-vol

Setting up clone for test-ldom with test-ldm-vol storage
creating domain
network
storage
adding disk to test-ldom LDOM
set autoboot var
binding
LDOM test-ldom started
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
test-ldom          active     ——  5013    2     4G       0.0%  0s

That was fast. Lets have a look and see how it looks:

# ldm list -e test-ldom
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
test-ldom          active     -t—-  5013    2     4G        50%  8m

SOFTSTATE
OpenBoot Running

UUID
70ba2009-37a1-ec70-a62e-f084b30057ca

MAC
00:14:4f:fb:7d:e7

HOSTID
0x84fb7de7

CONTROL
failure-policy=ignore
extended-mapin-space=off
cpu-arch=native

DEPENDENCY
master=

CORE
CID    CPUSET
14     (112, 113)

VCPU
VID    PID    CID    UTIL STRAND
0      112    14     100%   100%
1      113    14     0.0%   100%

MEMORY
RA               PA               SIZE
0x5f800000       0xe5f800000      4G

CONSTRAINT
threading=max-throughput

VARIABLES
auto-boot?=false

NETWORK
NAME             SERVICE                     ID   DEVICE     MAC               MODE   PVID VID
MTU   LINKPROP
vnet1            primary-vsw0@primary        0    network@0  00:14:4f:fb:f0:89        1
1500

DISK
NAME             VOLUME                      TOUT ID   DEVICE  SERVER         MPGROUP
vdisk0           test-ldm-vol@primary-vds0          0    disk@0  primary

VLDCC
NAME             SERVICE                     DESC
ds               primary-vldc0@primary       domain-services

VCONS
NAME             SERVICE                     PORT
test-ldom          primary-vcc0@primary        5013

Looks great. The virtual console primary-vcc0@primary is on port 5013 meaning this is the 14th LDOM on this system. Using this basic script you can create dozens of LDOMs with the golden image I created in the earlier steps. I would boot the newly created LDOMs to single user mode and change the hostname and ip address before going multiuser. Of course this script can be expanded to feed it memory and cpu values to give it more flexibilty. Nothing stopping you from added more memory and CPU later. Depending on your version of Domain Manager, you can use Dynamic Reconfiguration to change resources on the fly.

Lets boot the LDOM to single user mode. So I need to get to the console which should get me the OBP ok prompt. At which point I will boot it to single user mode and change the hostname and IP address and possibly routing information if need be.

# telnet localhost 5013
Trying 127.0.0.1…
Connected to localhost.
Escape character is '^]'.

Connecting to console "test-ldom" in group "test-ldom" ….
Press ~? for control options ..

{0} ok

{0} ok boot -vs
Boot device: /virtual-devices@100/channel-devices@200/disk@0  File and args: -vs
module /platform/sun4v/kernel/sparcv9/unix: text at [0x1000000, 0x10c1c1d] data at 0x1800000
module /platform/sun4v/kernel/sparcv9/genunix: text at [0x10c1c20, 0x12a6b77] data at 0x1935f40
module /platform/sun4v/kernel/misc/sparcv9/platmod: text at [0x12a6b78, 0x12a6b8f] data at 0x198d598
module /platform/sun4v/kernel/cpu/sparcv9/SPARC-T4: text at [0x12a6b90, 0x12ad04f] data at 0x198dcc0
SunOS Release 5.10 Version Generic_147440-01 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
Ethernet address = 0:14:4f:fb:7d:e7
mem = 4194304K (0x100000000)
avail mem = 3821568000
root nexus = SPARC T4-4
pseudo0 at root
pseudo0 is /pseudo
scsi_vhci0 at root
scsi_vhci0 is /scsi_vhci
virtual-device: cnex0
cnex0 is /virtual-devices@100/channel-devices@200
vdisk@0 is online using ldc@14,0
channel-device: vdc0
vdc0 is /virtual-devices@100/channel-devices@200/disk@0
root on rpool/ROOT/s10s_u10wos_17b fstype zfs
pseudo-device: dld0
dld0 is /pseudo/dld@0
cpu0: SPARC-T4 (chipid 0, clock 2998 MHz)
cpu1: SPARC-T4 (chipid 0, clock 2998 MHz)
iscsi0 at root
iscsi0 is /iscsi
Booting to milestone "milestone/single-user:default".
pseudo-device: zfs0
zfs0 is /pseudo/zfs@0
WARNING: vnet0 has duplicate address 010.238.198.220 (in use by 00:14:4f:fa:68:fa); disabled
Nov 30 08:17:45 svc.startd[10]: svc:/network/physical:default: Method "/lib/svc/method/net-physical" fai
led with exit status 96.
Nov 30 08:17:45 svc.startd[10]: network/physical:default misconfigured: transitioned to maintenance (see
'svcs -xv' for details)
Hostname: rocker
pseudo-device: devinfo0
devinfo0 is /pseudo/devinfo@0
pseudo-device: pseudo1
pseudo1 is /pseudo/zconsnex@1
pseudo-device: lockstat0
lockstat0 is /pseudo/lockstat@0
pseudo-device: fcode0
fcode0 is /pseudo/fcode@0
pseudo-device: llc10
llc10 is /pseudo/llc1@0
pseudo-device: lofi0
lofi0 is /pseudo/lofi@0
pseudo-device: trapstat0
trapstat0 is /pseudo/trapstat@0
pseudo-device: fbt0
fbt0 is /pseudo/fbt@0
pseudo-device: profile0
profile0 is /pseudo/profile@0
pseudo-device: systrace0
systrace0 is /pseudo/systrace@0
pseudo-device: sdt0
sdt0 is /pseudo/sdt@0
pseudo-device: fasttrap0
fasttrap0 is /pseudo/fasttrap@0
pseudo-device: ntwdt0
ntwdt0 is /pseudo/ntwdt@0
pseudo-device: mdesc0
mdesc0 is /pseudo/mdesc@0
pseudo-device: ds_snmp0
ds_snmp0 is /pseudo/ds_snmp@0
pseudo-device: ds_pri0
ds_pri0 is /pseudo/ds_pri@0
pseudo-device: bmc0
bmc0 is /pseudo/bmc@0
pseudo-device: fcsm0
fcsm0 is /pseudo/fcsm@0
pseudo-device: fssnap0
fssnap0 is /pseudo/fssnap@0
pseudo-device: winlock0
winlock0 is /pseudo/winlock@0
pseudo-device: vol0
vol0 is /pseudo/vol@0
pseudo-device: pm0
pm0 is /pseudo/pm@0
pseudo-device: pool0
pool0 is /pseudo/pool@0
dump on /dev/zvol/dsk/rpool/dump size 1536 MB
Requesting System Maintenance Mode
SINGLE USER MODE

Root password for system maintenance (control-d to bypass):

Ok, we have logged in to single usermode using the password we configured in the golden image. Lets make some minore changes. These are basic but cleanly boot your system

# echo "23.45.66.111   test-ldom test-ldom.mydomain.com" > /etc/hosts

# echo test-ldom > /etc/hostname.vnet0
# echo test-ldom > /etc/nodename
# hostname test-ldom

Ok, that's enough for a basic setup. Should boot multi-user happily and let us install…. whatever you want to run.

^D
# svc.startd: Returning to milestone all.
pseudo-device: drctl0
drctl0 is /pseudo/drctl@0
pseudo-device: ramdisk1024
ramdisk1024 is /pseudo/ramdisk@1024
pseudo-device: dtrace0
dtrace0 is /pseudo/dtrace@0
pseudo-device: fcp0
fcp0 is /pseudo/fcp@0
Nov 30 08:22:23 ldmad: agent agent-device registered
Nov 30 08:22:23 ldmad: agent agent-system registered
Nov 30 08:22:23 ldmad: agent agent-dio registered
syslogd: line 24: WARNING: loghost could not be resolved

test-ldom console login: root
Password:

Nov 30 08:22:36 test-ldom pseudo: pseudo-device: devinfo0
Nov 30 08:22:36 test-ldom genunix: devinfo0 is /pseudo/devinfo@0
Nov 30 08:22:36 test-ldom login: ROOT LOGIN /dev/console
Last login: Thu Nov 29 13:46:20 on console
Oracle Corporation      SunOS 5.10      Generic Patch   January 2005
#

From here, you can add more disk, 300 IP addresses, even build branded zones if you need to. God forbid. And if you are really keen, you can do this:

As I said, this script can easily be modified to set the number of CPU's or memory settings as well. Through dynamic reconfiguration, this can be done on the fly later on too.

Solaris/LDOMs: using PCIe direct IO with LDOMs

Recent versions of logical domains (or Oracle VM for SPARC) allow you to assign single PCIe devices to a guest LDOM so IO from that LDOM does not have to go through the primary domain. I am setting this up for 2 FC HBA on a T4-2 system with two domains, one for prod and one for test. Assigning DIO devices to a guest domain (which then becomes an IO-Domain) will prevent you from doing live migration of this domain and it will also provide a new dependency to the primary domain because if the primary goes down or reboots, so does the PCI bus and with it the access to the HBA. But since we also boot from a ZFS provided by the primary domain, this dependency was already there as well. Another option would be to assign a whole PCIe bus to a guest domain (making it a so-called root domain) but extra caution needs to be taken if the primary domains boots from a disk controller attached to the PCIe bus to be shared. And some more thought needs to be put into your networking configuration as well.

The whole process is documented well, this post basically repeats the steps that I have taken and adds the multipath configuration from the guest domain.
The first step is to identify the device names of these FC adapters using ldm list-io from the primary domain (abbreviated output below).

root@primary:~# ldm list-io -l

NAME TYPE BUS DOMAIN STATUS

---- ---- --- ------ ------

pci_0 BUS pci_0 primary

[pci@400]

niu_0 NIU niu_0 primary

[niu@480]

pci_1 BUS pci_1 primary

[pci@500]

niu_1 NIU niu_1 primary

[niu@580]

/SYS/MB/PCIE0 PCIE pci_0 primary OCC

[pci@400/pci@2/pci@0/pci@8]

SUNW,qlc@0/fp/disk

SUNW,qlc@0/fp@0,0

SUNW,qlc@0,1/fp/disk

SUNW,qlc@0,1/fp@0,0

/SYS/MB/PCIE1 PCIE pci_1 primary OCC

[pci@500/pci@2/pci@0/pci@a]

SUNW,qlc@0/fp/disk

SUNW,qlc@0/fp@0,0

SUNW,qlc@0,1/fp/disk

SUNW,qlc@0,1/fp@0,0

So in my case, this is /SYS/MB/PCI0 and /SYS/MB/PCI1 of both PCI busses. So next we'll enable IO virtualization on both busses and remove the devices from the primary LDOM. The primary LDOM will need to be rebooted after this.

root@primary:~# ldm start-reconf primary

Initiating a delayed reconfiguration operation on the primary domain.

All configuration changes for other domains are disabled until the primary

domain reboots, at which time the new configuration for the primary domain

will also take effect.

 

root@primary:~# ldm set-io iov=on pci_0

------------------------------------------------------------------------------

Notice: The primary domain is in the process of a delayed reconfiguration.

Any changes made to the primary domain will only take effect after it reboots.

------------------------------------------------------------------------------

 

root@primary:~# ldm set-io iov=on pci_1

------------------------------------------------------------------------------

Notice: The primary domain is in the process of a delayed reconfiguration.

Any changes made to the primary domain will only take effect after it reboots.

------------------------------------------------------------------------------

 

root@primary:~# ldm remove-io /SYS/MB/PCIE0 primary

------------------------------------------------------------------------------

Notice: The primary domain is in the process of a delayed reconfiguration.

Any changes made to the primary domain will only take effect after it reboots.

------------------------------------------------------------------------------

 

root@primary:~# ldm remove-io /SYS/MB/PCIE1 primary

------------------------------------------------------------------------------

Notice: The primary domain is in the process of a delayed reconfiguration.

Any changes made to the primary domain will only take effect after it reboots.

------------------------------------------------------------------------------

 

root@primary:~# reboot -- -r

After the reboot the device(s) will show up as unassigned.

 

root@priamry:~# ldm list-io -l /SYS/MB/PCIE0

NAME TYPE BUS DOMAIN STATUS

---- ---- --- ------ ------

/SYS/MB/PCIE0 PCIE pci_0 OCC

[pci@400/pci@2/pci@0/pci@8]

SUNW,assigned-device@0

SUNW,assigned-device@0,1

And we can now assign these devices to the guest domains. They need to be stopped first (test was not installed at this point). The last steps sets up the dependency relationship to the primary LDOM so that the guests are also reset if the primary reboots.

root@primary:~# ldm stop-domain LDOM-prod

LDOM LDOM-prod stopped

 

root@primary:~# ldm stop-domain LDOM-test

Remote graceful shutdown or reboot capability is not available on LDOM-test

LDOM LDOM-test stopped

 

root@primary:~# ldm add-io /SYS/MB/PCIE0 LDOM-prod

root@primary:~# ldm add-io /SYS/MB/PCIE1 LDOM-test

root@primary:~# ldm set-domain failure-policy=reset primary

root@primary:~# ldm set-domain master=primary LDOM-prod

root@primary:~# ldm set-domain master=primary LDOM-test

Last step is to boot the guest back up, verify that the device is available there and set up multipathing.

root@primary:~# ldm start-domain LDOM-prod

LDOM LDOM-prod started

 

root@primary:~# telnet localhost 5001

Trying 127.0.0.1...

Connected to localhost.

Escape character is '^]'.

 

Connecting to console "LDOM-prod" in group "LDOM-prod" ....

Press ~? for control options ..

[... console login ...]

 

root@LDOM-prod:~# prtdiag -v

System Configuration: Oracle Corporation sun4v SPARC T4-2

Memory size: 204800 Megabytes

================================ Virtual CPUs ================================

CPU ID Frequency Implementation Status

------ --------- ---------------------- -------

0 2848 MHz SPARC-T4 on-line

1 2848 MHz SPARC-T4 on-line

2 2848 MHz SPARC-T4 on-line

3 2848 MHz SPARC-T4 on-line

================================ IO Devices ================================

Slot + Bus Name + Model Speed

Status Type Path

----------------------------------------------------------------------------

PCIE0 PCIE SUNW,qlc-pciex1077,2532 QLE2562 5.0GTx4

/pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0

PCIE0 PCIE SUNW,qlc-pciex1077,2532 QLE2562 5.0GTx4

/pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0,1

 

root@LDOM-prod:~# stmsboot -e

WARNING: stmsboot operates on each supported multipath-capable controller

detected in a host. In your system, these controllers are

 

/pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0/fp@0,0

/pci@400/pci@2/pci@0/pci@8/SUNW,qlc@0,1/fp@0,0

 

If you do NOT wish to operate on these controllers, please quit stmsboot

and re-invoke with -D { fp | mpt | mpt_sas | pmcs} to specify which controllers you wish

to modify your multipathing configuration for.

 

Do you wish to continue? [y/n] (default: y) y

WARNING: This operation will require a reboot.

Do you want to continue ? [y/n] (default: y) y

The changes will come into effect after rebooting the system.

Reboot the system now ? [y/n] (default: y) y

And after that we can use the FC HBA directly from our LDOM with multipathing.

Solaris/Shell-scripts: Timestamps for Log Files

There are two common occasions when you might want to get a timestamp

• If you want to create a logfile called "myapp_log.11.Mar.2007″
• If you want to write to a logfile with "myapp: 11 Mar 2007 22:14:44: Something Happened"

Either way, you want to get the current date, in the format you prefer – for example, it's easier if a filename doesn't include spaces.

For the purposes of this article, though for no particular reason, I am assuming that the current time is 10:14:44 PM on Sunday the 11th March 2007.

The tool to use is, naturally enough, called "date". It has a bucket-load of switches, but first, we'll deal with how to use them. For the full list, see the man page ("man date"), though I'll cover some of the more generally useful ones below.

Setting the Date/Time

The first thing to note, is that date has two aspects: It can set the system clock:

# date 031122142007.44

will set the clock to 03 11 22 14 2007 44 – that is, 03=March, 11=11th day, 22 = 10pm, 14 = 14 minutes past the hour, 2007 = year 2007, 44 = 44 seconds past the minute.

Heck, I don't even know why I bothered to spell it out, it's obvious. Of course the year should come between the minutes and the seconds (ahem).

Getting the Date/Time

The more often used feature of the date command, is to find the current system date / time, and that is what we shall focus on here. It doesn't follow tradition, in that it uses the "+" and "%" symbols, instead of the "-" symbol, for its switches.

H = Hours, M = Minutes, S = Seconds, so:

$ date +%H:%M:%S

22:14:44

Which means that you can name a logfile like this:

#!/bin/sh

LOGFILE=/tmp/log_`date +%H%M%S`.log

echo Starting work > $LOGFILE

do_stuff >> $LOGFILE

do_more_stuff >> $LOGFILE

echo Finished >> $LOGFILE

This will create a logfile called /tmp/log_221444.log

You can also put useful information to the logfile:

#!/bin/sh

LOGFILE=/tmp/log_`date +%H%M%S`.log

echo `date +%H:%M:%S : Starting work > $LOGFILE

do_stuff >> $LOGFILE

echo "`date +%H:%M:%S : Done do_stuff" >> $LOGFILE

do_more_stuff >> $LOGFILE

echo "`date +%H:%M:%S : Done do_more_stuff" >> $LOGFILE

echo Finished >> $LOGFILE

This will produce a logfile along the lines of:

$ cat /tmp/log_221444.log

22:14:44: Starting work

do_stuff : Doing stuff, takes a short while

22:14:53: Done do_stuff

do_more_stuff : Doing more stuff, this is quite time consuming.

22:18:35: Done do_more_stuff

$

Counting the Seconds

UNIX has 1st Jan 1970 as a "special" date, the start of the system clock; GNU date will tell you how many seconds have elapsed since midnight on 1st Jan 1970:

$ date +%s

1173651284

Whilst this information is not very useful in itself, it may be useful to know how many seconds have elapsed between two events:

$ cat list.sh

#!/bin/sh

start=`date +%s`

ls -R $1 > /dev/null 2>&1

end=`date +%s`

 

diff=`expr $end - $start`

echo "Started at $start : Ended at $end"

echo "Elapsed time = $diff seconds"

$ ./list.sh /usr/share

Started at 1173651284 : Ended at 1173651290

Elapsed time = 6 seconds

$

For more useful switches, see the man page, but here are a few handy ones:

$ date "+%a %b %d" # (in the local language)

Sun Mar 11

$ date +%D # (show the full date)

03/11/07

$ date +%F # (In another format)

2007-03-11

$ date +%j # (how many days into the year)

070

$ date +%u # (day of the week)

7

$

Solaris: The RSTCHOWN parameter & Changing File Ownership

By default, in Solaris, the owner of the file cannot use the chown command to change the owner of the file or directory. There is also restriction in using chgrp command. The owner can only use chgrp command to change the group of the file to a group, which the owner belongs to.

However, this behavior can be changed by modifying /etc/system file by adding the following line:

set rstchown=0

You need to reboot your system after this change. Only the root can arbitrarily change ownership of the file whether or not this option is in effect.

Let's check the difference in system's behavior in following examples. We have the user account nameduser1 which is member of groups admin and dba. With default behavior when user1 is trying to change owner of the data.log file to user2 he gets the following message

$ chown user2 data.log
chown: data.log: Not owner

$ ls -l
total 12
-rw-rw-r-- 1 user1 admin 5345 May 11 05:53 data.log

However, user1 can change group of the data.log file to dba, because that is one of the two groups which he is a member of. But when user1 tries to change group of the data.log file to the manager his attempt will fail.

$ chgrp dba data.log

$ ls -l
total 12
-rw-rw-r-- 1 user1 dba 5345 May 11 05:53 data.log

$ chgrp manager data.log
chgrp: data.log: Not owner

Now, after the /etc/system file modification, user1 is free to change owner or group of his data.log file.

$ chown user2 data.log

$ ls -l
total 12
-rw-rw-r-- 1 user2 admin 5345 May 11 05:53 data.log

$ chgrp manager data.log

$ ls -l
total 12
-rw-rw-r-- 1 user1 manager 5345 May 11 05:53 data.log

There is possibility to change rstchown parameter on the fly without reboot using adb (mdb). This change will not persist across reboot though.

# adb -w -k /dev/ksyms /dev/mem
physmem 1f425
rstchown/D
rstchown:
rstchown: 1
rstchown/W 0
rstchown: 0x1 = 0x0
rstchown/D
rstchown:
rstchown: 0

You need to provide the name of the parameter you want to change together with valid option. In example above we printed current value assigned to rstchown by using /D and then we assigned new value by /W 0.

Solaris 11: Activate FASTREBOOT to significantly reduce boot time

The Fast Reboot feature of Oracle Solaris 11 is supported on both SPARC and x86 platforms. The Fast Reboot feature implements an in-kernel boot loader that loads the kernel into memory and then switches to that kernel, so that the reboot process occurs within seconds.

For x86 systems:

Fast reboot, bypassing firmware and boot loader. The new kernel will be loaded into memory by the running kernel, and control will be transferred to the newly loaded kernel. If disk or kernel arguments are specified, they must be specified before other boot arguments.

For SPARC systems:

Speeds up rebooting by skipping some POST tests.

The Fast Reboot feature is controlled by SMF and implemented through a boot configuration service, svc:/system/boot-config. The boot-config service provides a means for setting or changing the default boot parameters.

The fastreboot_default property of the boot-config service enables an automatic fast reboot of the system when either the reboot or the init 6 command is used. When the config/fastreboot_default property is set to true, the system automatically performs a fast reboot, without the need to use the reboot -f command. By default, this property's value is set to true on an x86 based system and false on a SPARC based system.

 The svc:/system/boot-config:default service consists of the following properties:

   config/fastreboot_default

   config/fastreboot_onpanic

These properties can be configured by using the svccfg and svcadm commands.

For example, to disable the default behavior of the fastreboot_onpanic property on an x86 based system, you would set the property's value to false, as shown here:

# svccfg -s "system/boot-config:default" setprop config/fastreboot_onpanic=false

# svcadm refresh svc:/system/boot-config:default

Changing one property's value does not affect the default behavior of the other property.

The following example shows how to make a fast reboot the default behavior on a SPARC based system by setting the boot-config SMF service property to true.

# svccfg -s "system/boot-config:default" setprop config/fastreboot_default=true

# svcadm refresh svc:/system/boot-config:default

Setting the property's value to true accelerates the reboot process, which enables systems that support the Fast Reboot feature to bypass certain POST tests. When the property is set to true, you can perform a fast reboot of the system without having to use the -f option with the reboot command.

To reboot a system that has the Fast Reboot feature enabled, without reconfiguring the boot-config service to disable the feature, use the -p option with the reboot command, as shown here:

# reboot -p

Solaris: how to edit/vi huge files (GBs)

Every now and then you end up with task to read or edit file that is huge. By huge I mean the size of the file at least 1GB, more likely several GBs.
You usually notice huge file by the following message when you try to open it using vi editor:

Not enough space in /var/tmp

The default directory /var/tmp for the vi editing buffer needs space equal to roughly twice the size of the file with which you are working, because vi uses extra lines for buffer manipulation.
This is how you choose another directory for vi's editing buffer:

$ cd /where_your_file_is_located

$ vi

:set directory=/tmp

:e filename

You can replace /tmp with any other directory containing enough space as a editing buffer. The filename is the name of your large file.
In case you just need to read file and do not plan to edit it, you can use one of the commands that use fewer resources like less, head or tail.
Other option would be to split the file either size-wise or line-wise.

$ split -l 10000 xyz.txt

 Each of new files will have 10000 lines. Or you can try

$ split -b 250m xyz.txt xyz.txt.split

which will create 250MB chunks out of the original file.

If you prefer to use vim over vi, you might try to press Ctrl-C when loading your file. Vim tries to read in the whole file initially to do things like syntax highlighting, number of lines in file and so on. Ctrl-C will cancel this and will only load what's needed to display on your screen.

Solaris 10: Unable to login into NonGlobal Zones (NGZ) after Patching

Yesterday I faced an issue, where in I was unable to login into NGZ's after kernel patching. Though the zlogin was working perfectly.

Initially I think the cause is ssh key, then I tried to login into the NGZ from the network, it was showing ssh connection refused.

I have checked the ssh services via zlogin. There were many services related to network which were not running including ssh. All services were depending on /system/sysidtool:net service which is in disbale mode. I tried to enable /system/sysidtool:net service but No luck.

# svcs -vx

# svcs -a | grep -i /system/sysidtool:net

# svcadm enable /system/sysidtool:net

# svcs -a | grep -i /system/sysidtool:net

# svcs -vx

Then I examine the logs for this service failure and found:

[ Aug 27 09:15:49 Method "start" exited with status 0 ]
[ Aug 27 09:36:58 Enabled. ]
[ Aug 27 09:37:01 Executing start method ("/lib/svc/method/sysidtool-net") ]
/etc/.UNCONFIGURED not found. System already configured, /lib/svc/method/sysidtool-net exiting.
[ Aug 27 09:37:01 Method "start" exited with status 0 ]
[ Aug 27 14:58:06 Enabled. ]
[ Aug 27 14:58:12 Executing start method ("/lib/svc/method/sysidtool-net") ]
ifconfig: status: SIOCGLIFFLAGS: fjgi0: no such interface
ifconfig: setifflags: SIOCGLIFFLAGS: fjgi0: no such interface
ifconfig: status: SIOCGLIFFLAGS: fjgi7: no such interface
ifconfig: setifflags: SIOCGLIFFLAGS: fjgi7: no such interface
Terminated

At this point of time I checked the Interfaces, but all were up and running fine in Global as well as in Non-global zone.

Then one particular line got my attention:

"/etc/.UNCONFIGURED not found. System already configured, /lib/svc/method/sysidtool-net exiting."

I checked /etc/.UNCONFIGURE with ls -la in the NGZ and found one file with name .UNCONFIGURE, which was of 0 "Zero" size.

I removed this file and restarted the NZG, all went in favor and all services started successfully. 

# cd /etc

# ls -la

# rm .UNCONFIGURE

# zoneadm -z zone-name reboot