VCS ON RHEL6–CONFIGURE & OPERATION-P4

VCS ON RHEL6–CONFIGURE & OPERATION-P4

ONLINE/OFFLINE SERVICE GROUP:

SWITCHING SERVICE GROUP:

RESOURCE ONLINE:

RESOURCE OFFLINE:

STOP VCS:

START VCS:

VCS CONFIGURATION:

ONLINE CONFIG:

OFFLINE CONFIG:

CONFIG EDITING/NEW CREATION:

What if we forget to save and shutdown the VCS

AUTOMATIC CONFIG BACKUP:

ONLINE/OFFLINE SERVICE GROUP:

[root@pr01 ~]# hagrp -offline Web-App -sys pr01

[root@pr01 ~]# hastatus -sum Web-App

-- SYSTEM STATE

-- System               State                Frozen

A  dr01                 RUNNING              0

A  pr01                 RUNNING              0

-- GROUP STATE

-- Group           System               Probed     AutoDisabled    State

B  NFS_APP1        dr01                 Y          N               ONLINE

B  NFS_APP1        pr01                 Y          N               OFFLINE

B  Web-App         dr01                 Y          N               STARTING|PARTIAL

B  Web-App         pr01                 Y          N               OFFLINE

-- RESOURCES FAILED

-- Group           Type                 Resource             System

D  Web-App         Apache               Web_Res              pr01

-- RESOURCES ONLINING

-- Group           Type            Resource             System               IState

F  Web-App         Apache          Web_Res              dr01                 W_ONLINE

[root@pr01 ~]# hagrp -offline Web-App -sys dr01

[root@pr01 ~]# hastatus -sum Web-App

-- SYSTEM STATE

-- System               State                Frozen

A  dr01                 RUNNING              0

A  pr01                 RUNNING              0

-- GROUP STATE

-- Group           System               Probed     AutoDisabled    State

B  NFS_APP1        dr01                 Y          N               ONLINE

B  NFS_APP1        pr01                 Y          N               OFFLINE

B  Web-App         dr01                 Y          N               STOPPING|PARTIAL

B  Web-App         pr01                 Y          N               OFFLINE

-- RESOURCES FAILED

-- Group           Type                 Resource             System

D  Web-App         Apache               Web_Res              pr01

-- RESOURCES ONLINING

-- Group           Type            Resource             System               IState

F  Web-App         Apache          Web_Res              dr01                 W_ONLINE_REVERSE_PROPAGATE

[root@pr01 ~]# hagrp -online Web-App -sys dr01

[root@pr01 ~]# hastatus -sum Web-App

-- SYSTEM STATE

-- System               State                Frozen

A  dr01                 RUNNING              0

A  pr01                 RUNNING              0

-- GROUP STATE

-- Group           System               Probed     AutoDisabled    State

B  NFS_APP1        dr01                 Y          N               ONLINE

B  NFS_APP1        pr01                 Y          N               OFFLINE

B  Web-App         dr01                 Y          N               STARTING|PARTIAL

B  Web-App         pr01                 Y          N               OFFLINE

-- RESOURCES FAILED

-- Group           Type                 Resource             System

D  Web-App         Apache               Web_Res              pr01

-- RESOURCES ONLINING

-- Group           Type            Resource             System               IState

F  Web-App         Apache          Web_Res              dr01                 W_ONLINE

[root@pr01 ~]#

SWITCHING SERVICE GROUP:

In switching vcs replicates the state of each resource on target system. If we switch a offline SG, then the resources will not brought online on target system.

[root@pr01 ~]# hagrp -switch NFS_APP1 -to pr01

[root@pr01 ~]# hastatus -sum |grep -i NFS_APP1

B  NFS_APP1        dr01                 Y          N               STOPPING|PARTIAL

B  NFS_APP1        pr01                 Y          N               OFFLINE

G  NFS_APP1        DiskGroup       NFS_DG1              dr01                 W_OFFLINE_PROPAGATE

[root@pr01 ~]#

[root@pr01 ~]# hastatus -sum |grep -i NFS_APP1

B  NFS_APP1        dr01                 Y          N               OFFLINE

B  NFS_APP1        pr01                 Y          N               ONLINE

RESOURCE ONLINE:

[root@pr01 ~]# hares -display Web_Res

[root@pr01 ~]# hares -online Service_IP -sys dr01

[root@pr01 ~]# hares -display Service_IP

Service_IP   State                 dr01       ONLINE

Service_IP   State                 pr01       OFFLINE

RESOURCE OFFLINE:

Taking resources offline causes the SG to become partially online, and availability of application service is affected.

If multiple resources need to be taken offline manually, then they must be taken offline in resource dependency tree order, from top to bottom.

[root@pr01 ~]# hares -offline NFS_MOUNT1 -sys pr01

STOP VCS:

[root@pr01 ~]# hastop –local

OR

[root@dr01 ~]# hastop -local -force

OR

[root@pr01 ~]# hastop -all –force

OR

[root@pr01 ~]# hastop –all

[root@pr01 ~]# hastatus -sum

VCS ERROR V-16-1-10600 Cannot connect to VCS engine

VCS WARNING V-16-1-11046 Local system not available

“-local” for current system

“-local-evacuate” causes the SG on pr01 to be migrated to dr01 and stops the vcs

  engine (HAD) on pr01

“-local –force” stops Had on pr01, leaving services running. Vcs continuous to

  run on dr01

“-all –force” stops had on both systems and leave the service running. HA

  Services are available to user but they are not protected and failover will

  not happen. We can see this live in below example.

[root@pr01 ~]# hastatus -sum |grep -i online

B  NFS_APP1        pr01                 Y          N               ONLINE

[root@pr01 ~]# df -kh /nfs-test/

Filesystem            Size  Used Avail Use% Mounted on

/dev/vx/dsk/nfs_dg/nfs-vol

                      9.7G   21M  9.1G   1% /nfs-test

[root@pr01 ~]# cd /nfs-test

[root@pr01 nfs-test]# hastop -all –force

[root@pr01 nfs-test]# touch abc

[root@pr01 nfs-test]# ls -l

total 0

-rw-r--r-- 1 root root  0 Mar 10 11:45 abc

drwxr-xr-x 2 root root 96 Mar  5 19:13 lost+found

[root@pr01 nfs-test]# df -kh

Filesystem            Size  Used Avail Use% Mounted on

/dev/sda2              13G  6.8G  5.2G  57% /

tmpfs                 497M     0  497M   0% /dev/shm

/dev/sda1             485M   37M  423M   8% /boot

.host:/               108G  104G  3.9G  97% /mnt/hgfs

tmpfs                 4.0K     0  4.0K   0% /dev/vx

/dev/vx/dsk/nfs_dg/nfs-vol

                      9.7G   21M  9.1G   1% /nfs-test

/dev/vx/dsk/apps_dg/apps_vol

                       38G   79M   36G   1% /apps

START VCS:

[root@dr01 ~]# hastart –onenode

[root@dr01 ~]# hastatus -sum

-- SYSTEM STATE

-- System               State                Frozen

A  dr01                 RUNNING              0

A  pr01                 UNKNOWN              0

-- GROUP STATE

-- Group           System               Probed     AutoDisabled    State

B  NFS_APP1        dr01                 Y          N               ONLINE

B  NFS_APP1        pr01                 Y          Y               OFFLINE

B  Web-App         dr01                 Y          N               PARTIAL

B  Web-App         pr01                 Y          Y               OFFLINE

-- RESOURCES NOT PROBED

-- Group           Type                 Resource             System

E  NFS_APP1        DiskGroup            NFS_DG1              pr01

E  NFS_APP1        IP                   NFS_IP1              pr01

E  NFS_APP1        Mount                NFS_MOUNT1           pr01

E  NFS_APP1        NFS                  NFS_SERVICE1         pr01

E  NFS_APP1        NFSRestart           NFS_RESTART1         pr01

E  NFS_APP1        NIC                  NFS_NIC1             pr01

E  NFS_APP1        Share                NFS_SHARE1           pr01

E  NFS_APP1        Volume               NFS_VOLUME1          pr01

E  Web-App         Apache               Web_Res              pr01

E  Web-App         DiskGroup            DG_Res               pr01

E  Web-App         IP                   Service_IP           pr01

E  Web-App         Mount                Mount_Res            pr01

E  Web-App         NIC                  Nic_Res              pr01

E  Web-App         Volume               Volume_Res           pr01

[root@pr01 ~]# hastart

[root@pr01 ~]# hastatus -sum

-- SYSTEM STATE

-- System               State                Frozen

A  dr01                 UNKNOWN              0

A  pr01                 RUNNING              0

-- GROUP STATE

-- Group           System               Probed     AutoDisabled    State

B  NFS_APP1        dr01                 Y          Y               OFFLINE

B  NFS_APP1        pr01                 Y          N               OFFLINE

B  Web-App         dr01                 Y          Y               OFFLINE

B  Web-App         pr01                 N          Y               OFFLINE

-- RESOURCES NOT PROBED

-- Group           Type                 Resource             System

VCS CONFIGURATION:

ONLINE/OFFLINE:

ONLINE CONFIG:

While HAD is running, via GUI/CLI/SHELL SCRIPTS

These methods change the cluster config in the memory. After finishing the config we can write the “in-memory” configuration to the “main.cf” file on disk to make the config persistent by saving the config.

OFFLINE CONFIG:

Used while editing config files manually, or by GUI to create, mofdify and test configuration.

Offline method requires to stop/restart to build the new config in memory.

CONFIG EDITING/NEW CREATION:

OPEN => CHANGE/CREATE => SAVE

Open the config for changes,

[root@pr01 ~]# haconf –makerw

Do the config editing or new creation………

Save the config,

[root@pr01 ~]# haconf -dump –makero

Editing/creation is done to “main.cf” file located at

[root@pr01 /]# more /etc/VRTSvcs/conf/config/main.cf

If the config is open, we cannot stop the VCS. “-force’ will override the warning and we may lose our configuration changes.

[root@pr01 /]# haconf -makerw

[root@pr01 /]# hastop -all

VCS WARNING V-16-1-10028 Cluster configuration is writable.  Must do 'haconf -dump -makero' or 'hastop -all -force'

[root@pr01 /]#

What if we forget to save and shutdown the VCS…?

Again we lose the changes.

To prevent this, we need

AUTOMATIC CONFIG BACKUP:

“Backupinterval” cluster attribute is set for automatic periodic backup. This will save the “in-memory” config to the “main.cf.autobackup” file provided the “Backupinterval” value set equal or greater than 3 minutes.

If there is no change within provided time interval, then no backup copy created.

[root@pr01 /]# haclus -modify BackupInterval 5

VCS WARNING V-16-1-11335 Configuration must be ReadWrite : Use haconf -makerw

[root@pr01 /]# haconf -makerw

[root@pr01 /]# haclus -modify BackupInterval 5

Do the changes or remain ideal….. forget about the tension to lose of config changes.

[root@pr01 /]# haconf -dump -makero