Edition 3
Mono-spaced Bold
To see the contents of the filemy_next_bestselling_novelin your current working directory, enter thecat my_next_bestselling_novelcommand at the shell prompt and press Enter to execute the command.
Press Enter to execute the command.Press Ctrl+Alt+F1 to switch to the first virtual terminal. Press Ctrl+Alt+F7 to return to your X-Windows session.
mono-spaced bold. For example:
File-related classes includefilesystemfor file systems,filefor files, anddirfor directories. Each class has its own associated set of permissions.
Choose → → from the main menu bar to launch Mouse Preferences. In the Buttons tab, click the Left-handed mouse check box and click to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).To insert a special character into a gedit file, choose → → from the main menu bar. Next, choose → from the Character Map menu bar, type the name of the character in the Search field and click . The character you sought will be highlighted in the Character Table. Double-click this highlighted character to place it in the Text to copy field and then click the button. Now switch back to your document and choose → from the gedit menu bar.
Mono-spaced Bold ItalicProportional Bold Italic
To connect to a remote machine using ssh, typesshat a shell prompt. If the remote machine isusername@domain.nameexample.comand your username on that machine is john, typessh john@example.com.Themount -o remountcommand remounts the named file system. For example, to remount thefile-system/homefile system, the command ismount -o remount /home.To see the version of a currently installed package, use therpm -qcommand. It will return a result as follows:package.package-version-release
Publican is a DocBook publishing system.
mono-spaced roman and presented thus:
books Desktop documentation drafts mss photos stuff svn books_tests Desktop1 downloads images notes scripts svgs
mono-spaced roman but add syntax highlighting as follows:
package org.jboss.book.jca.ex1; import javax.naming.InitialContext; public class ExClient { public static void main(String args[]) throws Exception { InitialContext iniCtx = new InitialContext(); Object ref = iniCtx.lookup("EchoBean"); EchoHome home = (EchoHome) ref; Echo echo = home.create(); System.out.println("Created Echo"); System.out.println("Echo.echo('Hello') = " + echo.echo("Hello")); } }
[1] It is hoped however, that having understood the concepts explained here, that the functionality of these tools will also be more readily understood.
[2] Even though Pacemaker also supports Heartbeat, the filesystems need to use the stack for messaging and membership and Corosync seems to be what they're standardizing on. Technically it would be possible for them to support Heartbeat as well, however there seems little interest in this.
Review and modify partitioning layout box.
/ (aka. root) partition. Wel'll take some back so we can use DRBD.
Web Server since we plan on using Apache. Don't enable updates yet, we'll do that (and install any extra software we need) later. After you click next, Fedora will begin installing.
su command. You will need to supply the password you entered earlier during the installation process.
[beekhof@pcmk-1 ~]$ su -
Password:
[root@pcmk-1 ~]#
[root@pcmk-1 ~]#ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:6f:e1:58 brd ff:ff:ff:ff:ff:ff inet 192.168.9.41/24 brd 192.168.9.255 scope global eth0 inet6 ::20c:29ff:fe6f:e158/64 scope global dynamic valid_lft 2591667sec preferred_lft 604467sec inet6 2002:57ae:43fc:0:20c:29ff:fe6f:e158/64 scope global dynamic valid_lft 2591990sec preferred_lft 604790sec inet6 fe80::20c:29ff:fe6f:e158/64 scope link valid_lft forever preferred_lft forever [root@pcmk-1 ~]# ping -c 1 www.google.com PING www.l.google.com (74.125.39.99) 56(84) bytes of data. 64 bytes from fx-in-f99.1e100.net (74.125.39.99): icmp_seq=1 ttl=56 time=16.7 ms --- www.l.google.com ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 20ms rtt min/avg/max/mdev = 16.713/16.713/16.713/0.000 ms [root@pcmk-1 ~]#/sbin/chkconfig network on[root@pcmk-1 ~]#
[root@pcmk-1 ~]# sed -i.bak "s/SELINUX=enforcing/SELINUX=permissive/g" /etc/selinux/config [root@pcmk-1 ~]# /sbin/chkconfig --del iptables [root@pcmk-1 ~]# service iptables stop iptables: Flushing firewall rules: [ OK ] iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Unloading modules: [ OK ]
May 4 19:30:54 pcmk-1 setroubleshoot: SELinux is preventing /usr/sbin/corosync "getattr" access on /. For complete SELinux messages. run sealert -l 6e0d4384-638e-4d55-9aaf-7dac011f29c1 May 4 19:30:54 pcmk-1 setroubleshoot: SELinux is preventing /usr/sbin/corosync "getattr" access on /. For complete SELinux messages. run sealert -l 6e0d4384-638e-4d55-9aaf-7dac011f29c1
[root@pcmk-1 ~]#sed -i.bak "s/enabled=0/enabled=1/g" /etc/yum.repos.d/fedora.repo[root@pcmk-1 ~]#sed -i.bak "s/enabled=0/enabled=1/g" /etc/yum.repos.d/fedora-updates.repo[root@pcmk-1 ~]#yum install -y pacemaker corosyncLoaded plugins: presto, refresh-packagekit fedora/metalink | 22 kB 00:00 fedora-debuginfo/metalink | 16 kB 00:00 fedora-debuginfo | 3.2 kB 00:00 fedora-debuginfo/primary_db | 1.4 MB 00:04 fedora-source/metalink | 22 kB 00:00 fedora-source | 3.2 kB 00:00 fedora-source/primary_db | 3.0 MB 00:05 updates/metalink | 26 kB 00:00 updates | 2.6 kB 00:00 updates/primary_db | 1.1 kB 00:00 updates-debuginfo/metalink | 18 kB 00:00 updates-debuginfo | 2.6 kB 00:00 updates-debuginfo/primary_db | 1.1 kB 00:00 updates-source/metalink | 25 kB 00:00 updates-source | 2.6 kB 00:00 updates-source/primary_db | 1.1 kB 00:00 Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package corosync.x86_64 0:1.2.1-1.fc13 set to be updated --> Processing Dependency: corosynclib = 1.2.1-1.fc13 for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libquorum.so.4(COROSYNC_QUORUM_1.0)(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libvotequorum.so.4(COROSYNC_VOTEQUORUM_1.0)(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libcpg.so.4(COROSYNC_CPG_1.0)(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libconfdb.so.4(COROSYNC_CONFDB_1.0)(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libcfg.so.4(COROSYNC_CFG_0.82)(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libpload.so.4(COROSYNC_PLOAD_1.0)(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: liblogsys.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libconfdb.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libcoroipcc.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libcpg.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libquorum.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libcoroipcs.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libvotequorum.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libcfg.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libtotem_pg.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 --> Processing Dependency: libpload.so.4()(64bit) for package: corosync-1.2.1-1.fc13.x86_64 ---> Package pacemaker.x86_64 0:1.1.5-1.fc13 set to be updated --> Processing Dependency: heartbeat >= 3.0.0 for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: net-snmp >= 5.4 for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: resource-agents for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: cluster-glue for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libnetsnmp.so.20()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libcrmcluster.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libpengine.so.3()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libnetsnmpagent.so.20()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libesmtp.so.5()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libstonithd.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libhbclient.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libpils.so.2()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libpe_status.so.2()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libnetsnmpmibs.so.20()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libnetsnmphelpers.so.20()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libcib.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libccmclient.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libstonith.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: liblrm.so.2()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libtransitioner.so.1()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libpe_rules.so.2()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libcrmcommon.so.2()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Processing Dependency: libplumb.so.2()(64bit) for package: pacemaker-1.1.5-1.fc13.x86_64 --> Running transaction check ---> Package cluster-glue.x86_64 0:1.0.2-1.fc13 set to be updated --> Processing Dependency: perl-TimeDate for package: cluster-glue-1.0.2-1.fc13.x86_64 --> Processing Dependency: libOpenIPMIutils.so.0()(64bit) for package: cluster-glue-1.0.2-1.fc13.x86_64 --> Processing Dependency: libOpenIPMIposix.so.0()(64bit) for package: cluster-glue-1.0.2-1.fc13.x86_64 --> Processing Dependency: libopenhpi.so.2()(64bit) for package: cluster-glue-1.0.2-1.fc13.x86_64 --> Processing Dependency: libOpenIPMI.so.0()(64bit) for package: cluster-glue-1.0.2-1.fc13.x86_64 ---> Package cluster-glue-libs.x86_64 0:1.0.2-1.fc13 set to be updated ---> Package corosynclib.x86_64 0:1.2.1-1.fc13 set to be updated --> Processing Dependency: librdmacm.so.1(RDMACM_1.0)(64bit) for package: corosynclib-1.2.1-1.fc13.x86_64 --> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: corosynclib-1.2.1-1.fc13.x86_64 --> Processing Dependency: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: corosynclib-1.2.1-1.fc13.x86_64 --> Processing Dependency: libibverbs.so.1()(64bit) for package: corosynclib-1.2.1-1.fc13.x86_64 --> Processing Dependency: librdmacm.so.1()(64bit) for package: corosynclib-1.2.1-1.fc13.x86_64 ---> Package heartbeat.x86_64 0:3.0.0-0.7.0daab7da36a8.hg.fc13 set to be updated --> Processing Dependency: PyXML for package: heartbeat-3.0.0-0.7.0daab7da36a8.hg.fc13.x86_64 ---> Package heartbeat-libs.x86_64 0:3.0.0-0.7.0daab7da36a8.hg.fc13 set to be updated ---> Package libesmtp.x86_64 0:1.0.4-12.fc12 set to be updated ---> Package net-snmp.x86_64 1:5.5-12.fc13 set to be updated --> Processing Dependency: libsensors.so.4()(64bit) for package: 1:net-snmp-5.5-12.fc13.x86_64 ---> Package net-snmp-libs.x86_64 1:5.5-12.fc13 set to be updated ---> Package pacemaker-libs.x86_64 0:1.1.5-1.fc13 set to be updated ---> Package resource-agents.x86_64 0:3.0.10-1.fc13 set to be updated --> Processing Dependency: libnet.so.1()(64bit) for package: resource-agents-3.0.10-1.fc13.x86_64 --> Running transaction check ---> Package OpenIPMI-libs.x86_64 0:2.0.16-8.fc13 set to be updated ---> Package PyXML.x86_64 0:0.8.4-17.fc13 set to be updated ---> Package libibverbs.x86_64 0:1.1.3-4.fc13 set to be updated --> Processing Dependency: libibverbs-driver for package: libibverbs-1.1.3-4.fc13.x86_64 ---> Package libnet.x86_64 0:1.1.4-3.fc12 set to be updated ---> Package librdmacm.x86_64 0:1.0.10-2.fc13 set to be updated ---> Package lm_sensors-libs.x86_64 0:3.1.2-2.fc13 set to be updated ---> Package openhpi-libs.x86_64 0:2.14.1-3.fc13 set to be updated ---> Package perl-TimeDate.noarch 1:1.20-1.fc13 set to be updated --> Running transaction check ---> Package libmlx4.x86_64 0:1.0.1-5.fc13 set to be updated --> Finished Dependency Resolution Dependencies Resolved
========================================================================================== Package Arch Version Repository Size ========================================================================================== Installing: corosync x86_64 1.2.1-1.fc13 fedora 136 k pacemaker x86_64 1.1.5-1.fc13 fedora 543 k Installing for dependencies: OpenIPMI-libs x86_64 2.0.16-8.fc13 fedora 474 k PyXML x86_64 0.8.4-17.fc13 fedora 906 k cluster-glue x86_64 1.0.2-1.fc13 fedora 230 k cluster-glue-libs x86_64 1.0.2-1.fc13 fedora 116 k corosynclib x86_64 1.2.1-1.fc13 fedora 145 k heartbeat x86_64 3.0.0-0.7.0daab7da36a8.hg.fc13 updates 172 k heartbeat-libs x86_64 3.0.0-0.7.0daab7da36a8.hg.fc13 updates 265 k libesmtp x86_64 1.0.4-12.fc12 fedora 54 k libibverbs x86_64 1.1.3-4.fc13 fedora 42 k libmlx4 x86_64 1.0.1-5.fc13 fedora 27 k libnet x86_64 1.1.4-3.fc12 fedora 49 k librdmacm x86_64 1.0.10-2.fc13 fedora 22 k lm_sensors-libs x86_64 3.1.2-2.fc13 fedora 37 k net-snmp x86_64 1:5.5-12.fc13 fedora 295 k net-snmp-libs x86_64 1:5.5-12.fc13 fedora 1.5 M openhpi-libs x86_64 2.14.1-3.fc13 fedora 135 k pacemaker-libs x86_64 1.1.5-1.fc13 fedora 264 k perl-TimeDate noarch 1:1.20-1.fc13 fedora 42 k resource-agents x86_64 3.0.10-1.fc13 fedora 357 k Transaction Summary ========================================================================================= Install 21 Package(s) Upgrade 0 Package(s) Total download size: 5.7 M Installed size: 20 M Downloading Packages: Setting up and reading Presto delta metadata updates-testing/prestodelta | 164 kB 00:00 fedora/prestodelta | 150 B 00:00 Processing delta metadata Package(s) data still to download: 5.7 M (1/21): OpenIPMI-libs-2.0.16-8.fc13.x86_64.rpm | 474 kB 00:00 (2/21): PyXML-0.8.4-17.fc13.x86_64.rpm | 906 kB 00:01 (3/21): cluster-glue-1.0.2-1.fc13.x86_64.rpm | 230 kB 00:00 (4/21): cluster-glue-libs-1.0.2-1.fc13.x86_64.rpm | 116 kB 00:00 (5/21): corosync-1.2.1-1.fc13.x86_64.rpm | 136 kB 00:00 (6/21): corosynclib-1.2.1-1.fc13.x86_64.rpm | 145 kB 00:00 (7/21): heartbeat-3.0.0-0.7.0daab7da36a8.hg.fc13.x86_64.rpm | 172 kB 00:00 (8/21): heartbeat-libs-3.0.0-0.7.0daab7da36a8.hg.fc13.x86_64.rpm | 265 kB 00:00 (9/21): libesmtp-1.0.4-12.fc12.x86_64.rpm | 54 kB 00:00 (10/21): libibverbs-1.1.3-4.fc13.x86_64.rpm | 42 kB 00:00 (11/21): libmlx4-1.0.1-5.fc13.x86_64.rpm | 27 kB 00:00 (12/21): libnet-1.1.4-3.fc12.x86_64.rpm | 49 kB 00:00 (13/21): librdmacm-1.0.10-2.fc13.x86_64.rpm | 22 kB 00:00 (14/21): lm_sensors-libs-3.1.2-2.fc13.x86_64.rpm | 37 kB 00:00 (15/21): net-snmp-5.5-12.fc13.x86_64.rpm | 295 kB 00:00 (16/21): net-snmp-libs-5.5-12.fc13.x86_64.rpm | 1.5 MB 00:01 (17/21): openhpi-libs-2.14.1-3.fc13.x86_64.rpm | 135 kB 00:00 (18/21): pacemaker-1.1.5-1.fc13.x86_64.rpm | 543 kB 00:00 (19/21): pacemaker-libs-1.1.5-1.fc13.x86_64.rpm | 264 kB 00:00 (20/21): perl-TimeDate-1.20-1.fc13.noarch.rpm | 42 kB 00:00 (21/21): resource-agents-3.0.10-1.fc13.x86_64.rpm | 357 kB 00:00 ---------------------------------------------------------------------------------------- Total 539 kB/s | 5.7 MB 00:10 warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID e8e40fde: NOKEY fedora/gpgkey | 3.2 kB 00:00 ... Importing GPG key 0xE8E40FDE "Fedora (13) <fedora@fedoraproject.org%gt;" from /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-x86_64
Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : lm_sensors-libs-3.1.2-2.fc13.x86_64 1/21 Installing : 1:net-snmp-libs-5.5-12.fc13.x86_64 2/21 Installing : 1:net-snmp-5.5-12.fc13.x86_64 3/21 Installing : openhpi-libs-2.14.1-3.fc13.x86_64 4/21 Installing : libibverbs-1.1.3-4.fc13.x86_64 5/21 Installing : libmlx4-1.0.1-5.fc13.x86_64 6/21 Installing : librdmacm-1.0.10-2.fc13.x86_64 7/21 Installing : corosync-1.2.1-1.fc13.x86_64 8/21 Installing : corosynclib-1.2.1-1.fc13.x86_64 9/21 Installing : libesmtp-1.0.4-12.fc12.x86_64 10/21 Installing : OpenIPMI-libs-2.0.16-8.fc13.x86_64 11/21 Installing : PyXML-0.8.4-17.fc13.x86_64 12/21 Installing : libnet-1.1.4-3.fc12.x86_64 13/21 Installing : 1:perl-TimeDate-1.20-1.fc13.noarch 14/21 Installing : cluster-glue-1.0.2-1.fc13.x86_64 15/21 Installing : cluster-glue-libs-1.0.2-1.fc13.x86_64 16/21 Installing : resource-agents-3.0.10-1.fc13.x86_64 17/21 Installing : heartbeat-libs-3.0.0-0.7.0daab7da36a8.hg.fc13.x86_64 18/21 Installing : heartbeat-3.0.0-0.7.0daab7da36a8.hg.fc13.x86_64 19/21 Installing : pacemaker-1.1.5-1.fc13.x86_64 20/21 Installing : pacemaker-libs-1.1.5-1.fc13.x86_64 21/21 Installed: corosync.x86_64 0:1.2.1-1.fc13 pacemaker.x86_64 0:1.1.5-1.fc13 Dependency Installed: OpenIPMI-libs.x86_64 0:2.0.16-8.fc13 PyXML.x86_64 0:0.8.4-17.fc13 cluster-glue.x86_64 0:1.0.2-1.fc13 cluster-glue-libs.x86_64 0:1.0.2-1.fc13 corosynclib.x86_64 0:1.2.1-1.fc13 heartbeat.x86_64 0:3.0.0-0.7.0daab7da36a8.hg.fc13 heartbeat-libs.x86_64 0:3.0.0-0.7.0daab7da36a8.hg.fc13 libesmtp.x86_64 0:1.0.4-12.fc12 libibverbs.x86_64 0:1.1.3-4.fc13 libmlx4.x86_64 0:1.0.1-5.fc13 libnet.x86_64 0:1.1.4-3.fc12 librdmacm.x86_64 0:1.0.10-2.fc13 lm_sensors-libs.x86_64 0:3.1.2-2.fc13 net-snmp.x86_64 1:5.5-12.fc13 net-snmp-libs.x86_64 1:5.5-12.fc13 openhpi-libs.x86_64 0:2.14.1-3.fc13 pacemaker-libs.x86_64 0:1.1.5-1.fc13 perl-TimeDate.noarch 1:1.20-1.fc13 resource-agents.x86_64 0:3.0.10-1.fc13 Complete! [root@pcmk-1 ~]#
ping -c 3 192.168.122.102
[root@pcmk-1 ~]# ping -c 3 192.168.122.102
PING 192.168.122.102 (192.168.122.102) 56(84) bytes of data.
64 bytes from 192.168.122.102: icmp_seq=1 ttl=64 time=0.343 ms
64 bytes from 192.168.122.102: icmp_seq=2 ttl=64 time=0.402 ms
64 bytes from 192.168.122.102: icmp_seq=3 ttl=64 time=0.558 ms
--- 192.168.122.102 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.343/0.434/0.558/0.092 ms
grep pcmk /etc/hosts
[root@pcmk-1 ~]# grep pcmk /etc/hosts
192.168.122.101 pcmk-1.clusterlabs.org pcmk-1
192.168.122.102 pcmk-2.clusterlabs.org pcmk-2
ping -c 3 pcmk-2
[root@pcmk-1 ~]# ping -c 3 pcmk-2
PING pcmk-2.clusterlabs.org (192.168.122.101) 56(84) bytes of data.
64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=1 ttl=64 time=0.164 ms
64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=2 ttl=64 time=0.475 ms
64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=3 ttl=64 time=0.186 ms
--- pcmk-2.clusterlabs.org ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.164/0.275/0.475/0.141 ms
[root@pcmk-1 ~]#ssh-keygen -t dsa -f ~/.ssh/id_dsa -N ""Generating public/private dsa key pair. Your identification has been saved in /root/.ssh/id_dsa. Your public key has been saved in /root/.ssh/id_dsa.pub. The key fingerprint is: 91:09:5c:82:5a:6a:50:08:4e:b2:0c:62:de:cc:74:44 root@pcmk-1.clusterlabs.org The key's randomart image is: +--[ DSA 1024]----+ |==.ooEo.. | |X O + .o o | | * A + | | + . | | . S | | | | | | | | | +-----------------+ [root@pcmk-1 ~]#cp .ssh/id_dsa.pub .ssh/authorized_keys[root@pcmk-1 ~]#
[root@pcmk-1 ~]#scp -r .ssh pcmk-2:The authenticity of host 'pcmk-2 (192.168.122.102)' can't be established. RSA key fingerprint is b1:2b:55:93:f1:d9:52:2b:0f:f2:8a:4e:ae:c6:7c:9a. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'pcmk-2,192.168.122.102' (RSA) to the list of known hosts. root@pcmk-2's password: id_dsa.pub 100% 616 0.6KB/s 00:00 id_dsa 100% 672 0.7KB/s 00:00 known_hosts 100% 400 0.4KB/s 00:00 authorized_keys 100% 616 0.6KB/s 00:00 [root@pcmk-1 ~]#ssh pcmk-2 -- uname -npcmk-2 [root@pcmk-1 ~]#
[root@pcmk-1 ~]#uname -npcmk-1.clusterlabs.org [root@pcmk-1 ~]#dnsdomainnameclusterlabs.org
[root@pcmk-1 ~]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=pcmk-1.clusterlabs.org
GATEWAY=192.168.122.1
[root@pcmk-1 ~]# sed -i.bak 's/\.[a-z].*//g' /etc/sysconfig/network
[root@pcmk-1 ~]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=pcmk-1
GATEWAY=192.168.122.1
[root@pcmk-1 ~]#source /etc/sysconfig/network[root@pcmk-1 ~]#hostname $HOSTNAME
[root@pcmk-1 ~]#uname -npcmk-1 [root@pcmk-1 ~]#dnsdomainnameclusterlabs.org
[root@pcmk-1 ~]#export ais_port=4000[root@pcmk-1 ~]#export ais_mcast=226.94.1.1
[root@pcmk-1 ~]# export ais_addr=`ip addr | grep "inet " | tail -n 1 | awk '{print $4}' | sed s/255/0/`
[root@pcmk-1 ~]# env | grep ais_
ais_mcast=226.94.1.1
ais_port=4000
ais_addr=192.168.122.0
[root@pcmk-1 ~]#cp /etc/corosync/corosync.conf.example /etc/corosync/corosync.conf[root@pcmk-1 ~]#sed -i.bak "s/.*mcastaddr:.*/mcastaddr:\ $ais_mcast/g" /etc/corosync/corosync.conf[root@pcmk-1 ~]#sed -i.bak "s/.*mcastport:.*/mcastport:\ $ais_port/g" /etc/corosync/corosync.conf[root@pcmk-1 ~]#sed -i.bak "s/.*bindnetaddr:.*/bindnetaddr:\ $ais_addr/g" /etc/corosync/corosync.conf
[root@pcmk-1 ~]# cat <<-END >>/etc/corosync/service.d/pcmk
service {
# Load the Pacemaker Cluster Resource Manager
name: pacemaker
ver: 1
}
END
[root@pcmk-1 ~]# for f in /etc/corosync/corosync.conf /etc/corosync/service.d/pcmk /etc/hosts; do scp $f pcmk-2:$f ; done
corosync.conf 100% 1528 1.5KB/s 00:00
hosts 100% 281 0.3KB/s 00:00
[root@pcmk-1 ~]#
[root@pcmk-1 ~]# /etc/init.d/corosync start
Starting Corosync Cluster Engine (corosync): [ OK ]
[root@pcmk-1 ~]#grep -e "corosync.*network interface" -e "Corosync Cluster Engine" -e "Successfully read main configuration file" /var/log/messagesAug 27 09:05:34 pcmk-1 corosync[1540]: [MAIN ] Corosync Cluster Engine ('1.1.0'): started and ready to provide service. Aug 27 09:05:34 pcmk-1 corosync[1540]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'. [root@pcmk-1 ~]#grep TOTEM /var/log/messagesAug 27 09:05:34 pcmk-1 corosync[1540]: [TOTEM ] Initializing transport (UDP/IP). Aug 27 09:05:34 pcmk-1 corosync[1540]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0). Aug 27 09:05:35 pcmk-1 corosync[1540]: [TOTEM ] The network interface [192.168.122.101] is now up. Aug 27 09:05:35 pcmk-1 corosync[1540]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
[root@pcmk-1 ~]# ssh pcmk-2 -- /etc/init.d/corosync start
Starting Corosync Cluster Engine (corosync): [ OK ]
[root@pcmk-1 ~]#
[root@pcmk-1 ~]# grep TOTEM /var/log/messages
Aug 27 09:05:34 pcmk-1 corosync[1540]: [TOTEM ] Initializing transport (UDP/IP).
Aug 27 09:05:34 pcmk-1 corosync[1540]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Aug 27 09:05:35 pcmk-1 corosync[1540]: [TOTEM ] The network interface [192.168.122.101] is now up.
Aug 27 09:05:35 pcmk-1 corosync[1540]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Aug 27 09:12:11 pcmk-1 corosync[1540]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
[root@pcmk-1 ~]# grep pcmk_startup /var/log/messages
Aug 27 09:05:35 pcmk-1 corosync[1540]: [pcmk ] info: pcmk_startup: CRM: Initialized
Aug 27 09:05:35 pcmk-1 corosync[1540]: [pcmk ] Logging: Initialized pcmk_startup
Aug 27 09:05:35 pcmk-1 corosync[1540]: [pcmk ] info: pcmk_startup: Maximum core file size is: 18446744073709551615
Aug 27 09:05:35 pcmk-1 corosync[1540]: [pcmk ] info: pcmk_startup: Service: 9
Aug 27 09:05:35 pcmk-1 corosync[1540]: [pcmk ] info: pcmk_startup: Local hostname: pcmk-1
[root@pcmk-1 ~]# /etc/init.d/pacemaker start
Starting Pacemaker Cluster Manager: [ OK ]
[root@pcmk-1 ~]# grep -e pacemakerd.*get_config_opt -e pacemakerd.*start_child -e "Starting Pacemaker" /var/log/messages
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found 'pacemaker' for option: name
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found '1' for option: ver
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Defaulting to 'no' for option: use_logd
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Defaulting to 'no' for option: use_mgmtd
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found 'on' for option: debug
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found 'yes' for option: to_logfile
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found '/var/log/corosync.log' for option: logfile
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found 'yes' for option: to_syslog
Feb 8 13:31:24 pcmk-1 pacemakerd: [13155]: info: get_config_opt: Found 'daemon' for option: syslog_facility
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: main: Starting Pacemaker 1.1.5 (Build: 31f088949239+): docbook-manpages publican ncurses trace-logging cman cs-quorum heartbeat corosync snmp libesmtp
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: start_child: Forked child 14022 for process stonith-ng
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: start_child: Forked child 14023 for process cib
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: start_child: Forked child 14024 for process lrmd
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: start_child: Forked child 14025 for process attrd
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: start_child: Forked child 14026 for process pengine
Feb 8 16:50:38 pcmk-1 pacemakerd: [13990]: info: start_child: Forked child 14027 for process crmd
[root@pcmk-1 ~]# ps axf
PID TTY STAT TIME COMMAND
2 ? S< 0:00 [kthreadd]
3 ? S< 0:00 \_ [migration/0]
... lots of processes ...
13990 ? S 0:01 pacemakerd
14022 ? Sa 0:00 \_ /usr/lib64/heartbeat/stonithd
14023 ? Sa 0:00 \_ /usr/lib64/heartbeat/cib
14024 ? Sa 0:00 \_ /usr/lib64/heartbeat/lrmd
14025 ? Sa 0:00 \_ /usr/lib64/heartbeat/attrd
14026 ? Sa 0:00 \_ /usr/lib64/heartbeat/pengine
14027 ? Sa 0:00 \_ /usr/lib64/heartbeat/crmd
[root@pcmk-1 ~]# grep ERROR: /var/log/messages | grep -v unpack_resources
[root@pcmk-1 ~]#
[root@pcmk-1 ~]#ssh pcmk-2 -- /etc/init.d/pacemaker startStarting Pacemaker Cluster Manager: [ OK ] [root@pcmk-1 ~]#crm_mon============ Last updated: Thu Aug 27 16:54:55 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 0 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ]
[root@pcmk-1 ~]# crm --help
usage:
crm [-D display_type]
crm [-D display_type] args
crm [-D display_type] [-f file]
Use crm without arguments for an interactive session.
Supply one or more arguments for a "single-shot" use.
Specify with -f a file which contains a script. Use '-' for
standard input or use pipe/redirection.
crm displays cli format configurations using a color scheme
and/or in uppercase. Pick one of "color" or "uppercase", or
use "-D color,uppercase" if you want colorful uppercase.
Get plain output by "-D plain". The default may be set in
user preferences (options).
Examples:
# crm -f stopapp2.cli
# crm < stopapp2.cli
# crm resource stop global_www
# crm status
--help option or consult the included man pages. Both sets of output are created from the tool, and so will always be in sync with each other and the tool itself.
--version option.
[root@pcmk-1 ~]#crm_mon --versionPacemaker 1.1.5 Written by Andrew Beekhof [root@pcmk-1 ~]#crm_mon --helpcrm_mon - Provides a summary of cluster's current state. Outputs varying levels of detail in a number of different formats. Usage: crm_mon mode [options] Options: -?, --help This text -$, --version Version information -V, --verbose Increase debug output Modes: -h, --as-html=value Write cluster status to the named file -w, --web-cgi Web mode with output suitable for cgi -s, --simple-status Display the cluster status once as a simple one line output (suitable for nagios) -S, --snmp-traps=value Send SNMP traps to this station -T, --mail-to=value Send Mail alerts to this user. See also --mail-from, --mail-host, --mail-prefix Display Options: -n, --group-by-node Group resources by node -r, --inactive Display inactive resources -f, --failcounts Display resource fail counts -o, --operations Display resource operation history -t, --timing-details Display resource operation history with timing details Additional Options: -i, --interval=value Update frequency in seconds -1, --one-shot Display the cluster status once on the console and exit -N, --disable-ncurses Disable the use of ncurses -d, --daemonize Run in the background as a daemon -p, --pid-file=value (Advanced) Daemon pid file location -F, --mail-from=value Mail alerts should come from the named user -H, --mail-host=value Mail alerts should be sent via the named host -P, --mail-prefix=value Subjects for mail alerts should start with this string -E, --external-agent=value A program to run when resource operations take place. -e, --external-recipient=value A recipient for your program (assuming you want the program to send something to someone). Examples: Display the cluster´s status on the console with updates as they occur: # crm_mon Display the cluster´s status on the console just once then exit: # crm_mon -1 Display your cluster´s status, group resources by node, and include inactive resources in the list: # crm_mon --group-by-node --inactive Start crm_mon as a background daemon and have it write the cluster´s status to an HTML file: # crm_mon --daemonize --as-html /path/to/docroot/filename.html Start crm_mon as a background daemon and have it send email alerts: # crm_mon --daemonize --mail-to user@example.com --mail-host mail.example.com Start crm_mon as a background daemon and have it send SNMP alerts: # crm_mon --daemonize --snmp-traps snmptrapd.example.com Report bugs to pacemaker@oss.clusterlabs.org
[root@pcmk-2 ~]# crm configure show
node pcmk-1
node pcmk-2
property $id="cib-bootstrap-options" \
dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2"
[root@pcmk-2 ~]# crm configure show xml
<?xml version="1.0" ?>
<cib admin_epoch="0" crm_feature_set="3.0.1" dc-uuid="pcmk-1" epoch="13" have-quorum="1" num_updates="7" validate-with="pacemaker-1.0">
<configuration>
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f"/>
<nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="openais"/>
<nvpair id="cib-bootstrap-options-expected-quorum-votes" name="expected-quorum-votes" value="2"/>
</cluster_property_set>
</crm_config>
<rsc_defaults/>
<op_defaults/>
<nodes>
<node id="pcmk-1" type="normal" uname="pcmk-1"/>
<node id="pcmk-2" type="normal" uname="pcmk-2"/>
</nodes>
<resources/>
<constraints/>
</configuration>
</cib>
[root@pcmk-1 ~]# crm_verify -L
crm_verify[2195]: 2009/08/27_16:57:12 ERROR: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
crm_verify[2195]: 2009/08/27_16:57:12 ERROR: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
crm_verify[2195]: 2009/08/27_16:57:12 ERROR: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
-V may provide more details
[root@pcmk-1 ~]#
crm configure property stonith-enabled=falsecrm_verify -L
stonith-enabled=false is completely inappropriate for a production cluster. We use it here to defer the discussion of its configuration which can differ widely from one installation to the next. See Chapter 9, Configure STONITH for information on why STONITH is important and details on how to configure it.
crm configure primitive ClusterIP ocf:heartbeat:IPaddr2 \params ip=192.168.122.101 cidr_netmask=32 \op monitor interval=30s
[root@pcmk-1 ~]# crm ra classes
heartbeat
lsb
ocf / heartbeat pacemaker
stonith
[root@pcmk-1 ~]#crm ra list ocf pacemakerClusterMon Dummy Stateful SysInfo SystemHealth controld ping pingd [root@pcmk-1 ~]#crm ra list ocf heartbeatAoEtarget AudibleAlarm ClusterMon Delay Dummy EvmsSCC Evmsd Filesystem ICP IPaddr IPaddr2 IPsrcaddr LVM LinuxSCSI MailTo ManageRAID ManageVE Pure-FTPd Raid1 Route SAPDatabase SAPInstance SendArp ServeRAID SphinxSearchDaemon Squid Stateful SysInfo VIPArip VirtualDomain WAS WAS6 WinPopup Xen Xinetd anything apache db2 drbd eDir88 iSCSILogicalUnit iSCSITarget ids iscsi ldirectord mysql mysql-proxy nfsserver oracle oralsnr pgsql pingd portblock rsyncd scsi2reservation sfex tomcat vmware [root@pcmk-1 ~]#
[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ [root@pcmk-1 ~]#crm_mon============ Last updated: Fri Aug 28 15:23:48 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1
[root@pcmk-1 ~]# crm resource status ClusterIP
resource ClusterIP is running on: pcmk-1
[root@pcmk-1 ~]#
[root@pcmk-1 ~]#ssh pcmk-1 -- /etc/init.d/pacemaker stopSignaling Pacemaker Cluster Manager to terminate: [ OK ] Waiting for cluster services to unload:. [ OK ] [root@pcmk-1 ~]#ssh pcmk-1 -- /etc/init.d/corosync stopStopping Corosync Cluster Engine (corosync): [ OK ] Waiting for services to unload: [ OK ] [root@pcmk-1 ~]#
[root@pcmk-2 ~]# crm_mon
============
Last updated: Fri Aug 28 15:27:35 2009
Stack: openais
Current DC: pcmk-2 - partition WITHOUT quorum
Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f
2 Nodes configured, 2 expected votes
1 Resources configured.
============
Online: [ pcmk-2 ]
OFFLINE: [ pcmk-1 ]
[root@pcmk-1 ~]#crm configure property no-quorum-policy=ignore[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore"
[root@pcmk-2 ~]# crm_mon
============
Last updated: Fri Aug 28 15:30:18 2009
Stack: openais
Current DC: pcmk-2 - partition WITHOUT quorum
Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f
2 Nodes configured, 2 expected votes
1 Resources configured.
============
Online: [ pcmk-2 ]
OFFLINE: [ pcmk-1 ]
ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2
[root@pcmk-1 ~]#/etc/init.d/corosync startStarting Corosync Cluster Engine (corosync): [ OK ] [root@pcmk-1 ~]#/etc/init.d/pacemaker startStarting Pacemaker Cluster Manager: [ OK ] [root@pcmk-1 ~]#crm_mon============ Last updated: Fri Aug 28 15:32:13 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1
crm configure rsc_defaults resource-stickiness=100
[root@pcmk-2 ~]# crm configure show
node pcmk-1
node pcmk-2
primitive ClusterIP ocf:heartbeat:IPaddr2 \
params ip="192.168.122.101" cidr_netmask="32" \
op monitor interval="30s"
property $id="cib-bootstrap-options" \
dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
cluster-infrastructure="openais" \
expected-quorum-votes="2" \
stonith-enabled="false" \
no-quorum-policy="ignore"
rsc_defaults $id="rsc-options" \
resource-stickiness="100"
[root@pcmk-1 ~]#ssh pcmk-1 -- /etc/init.d/pacemaker stopSignaling Pacemaker Cluster Manager to terminate: [ OK ] Waiting for cluster services to unload:. [ OK ] [root@pcmk-1 ~]#ssh pcmk-1 -- /etc/init.d/corosync stopStopping Corosync Cluster Engine (corosync): [ OK ] Waiting for services to unload: [ OK ] [root@pcmk-1 ~]#ssh pcmk-2 -- crm_mon -1============ Last updated: Fri Aug 28 15:39:38 2009 Stack: openais Current DC: pcmk-2 - partition WITHOUT quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ pcmk-2 ] OFFLINE: [ pcmk-1 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2
[root@pcmk-1 ~]#/etc/init.d/corosync startStarting Corosync Cluster Engine (corosync): [ OK ] [root@pcmk-1 ~]#/etc/init.d/pacemaker startStarting Pacemaker Cluster Manager: [ OK ] [root@pcmk-1 ~]#crm_mon============ Last updated: Fri Aug 28 15:41:23 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 1 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2
[11] If the data is corrupt, there is little point in continuing to make it available
[12] A common node fencing mechanism. Used to ensure data integrity by powering off “bad” nodes.
[13] Actually some would argue that two-node clusters are always pointless, but that is an argument for another time.
[14] It should be noted that Pacemaker’s definition of optimal may not always agree with that of a human’s. The order in which Pacemaker processes lists of resources and nodes create implicit preferences (required in order to create a stabile solution) in situations where the administrator had not explicitly specified some.
[root@ppcmk-1 ~]# yum install -y httpd
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 0:2.2.13-2.fc12 set to be updated
--> Processing Dependency: httpd-tools = 2.2.13-2.fc12 for package: httpd-2.2.13-2.fc12.x86_64
--> Processing Dependency: apr-util-ldap for package: httpd-2.2.13-2.fc12.x86_64
--> Processing Dependency: /etc/mime.types for package: httpd-2.2.13-2.fc12.x86_64
--> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.2.13-2.fc12.x86_64
--> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.2.13-2.fc12.x86_64
--> Running transaction check
---> Package apr.x86_64 0:1.3.9-2.fc12 set to be updated
---> Package apr-util.x86_64 0:1.3.9-2.fc12 set to be updated
---> Package apr-util-ldap.x86_64 0:1.3.9-2.fc12 set to be updated
---> Package httpd-tools.x86_64 0:2.2.13-2.fc12 set to be updated
---> Package mailcap.noarch 0:2.1.30-1.fc12 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================================
Package Arch Version Repository Size
=======================================================================================
Installing:
httpd x86_64 2.2.13-2.fc12 rawhide 735 k
Installing for dependencies:
apr x86_64 1.3.9-2.fc12 rawhide 117 k
apr-util x86_64 1.3.9-2.fc12 rawhide 84 k
apr-util-ldap x86_64 1.3.9-2.fc12 rawhide 15 k
httpd-tools x86_64 2.2.13-2.fc12 rawhide 63 k
mailcap noarch 2.1.30-1.fc12 rawhide 25 k
Transaction Summary
=======================================================================================
Install 6 Package(s)
Upgrade 0 Package(s)
Total download size: 1.0 M
Downloading Packages:
(1/6): apr-1.3.9-2.fc12.x86_64.rpm | 117 kB 00:00
(2/6): apr-util-1.3.9-2.fc12.x86_64.rpm | 84 kB 00:00
(3/6): apr-util-ldap-1.3.9-2.fc12.x86_64.rpm | 15 kB 00:00
(4/6): httpd-2.2.13-2.fc12.x86_64.rpm | 735 kB 00:00
(5/6): httpd-tools-2.2.13-2.fc12.x86_64.rpm | 63 kB 00:00
(6/6): mailcap-2.1.30-1.fc12.noarch.rpm | 25 kB 00:00
----------------------------------------------------------------------------------------
Total 875 kB/s | 1.0 MB 00:01
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : apr-1.3.9-2.fc12.x86_64 1/6
Installing : apr-util-1.3.9-2.fc12.x86_64 2/6
Installing : apr-util-ldap-1.3.9-2.fc12.x86_64 3/6
Installing : httpd-tools-2.2.13-2.fc12.x86_64 4/6
Installing : mailcap-2.1.30-1.fc12.noarch 5/6
Installing : httpd-2.2.13-2.fc12.x86_64 6/6
Installed:
httpd.x86_64 0:2.2.13-2.fc12
Dependency Installed:
apr.x86_64 0:1.3.9-2.fc12 apr-util.x86_64 0:1.3.9-2.fc12
apr-util-ldap.x86_64 0:1.3.9-2.fc12 httpd-tools.x86_64 0:2.2.13-2.fc12
mailcap.noarch 0:2.1.30-1.fc12
Complete!
[root@pcmk-1 ~]#
[root@pcmk-1 ~]# yum install -y wget
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.11.4-5.fc12 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================
Package Arch Version Repository Size
===========================================================================================
Installing:
wget x86_64 1.11.4-5.fc12 rawhide 393 k
Transaction Summary
===========================================================================================
Install 1 Package(s)
Upgrade 0 Package(s)
Total download size: 393 k
Downloading Packages:
wget-1.11.4-5.fc12.x86_64.rpm | 393 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : wget-1.11.4-5.fc12.x86_64 1/1
Installed:
wget.x86_64 0:1.11.4-5.fc12
Complete!
[root@pcmk-1 ~]#
[root@pcmk-1 ~]# cat <<-END >/var/www/html/index.html
<html>
<body>My Test Site - pcmk-1</body>
</html>
END
[root@pcmk-1 ~]#
[root@pcmk-2 ~]# cat <<-END >/var/www/html/index.html
<html>
<body>My Test Site - pcmk-2</body>
</html>
END
[root@pcmk-2 ~]#
<Location /server-status> SetHandler server-status Order deny,allow Deny from all Allow from 127.0.0.1 </Location>
[root@pcmk-1 ~]#crm configure primitive WebSite ocf:heartbeat:apache params configfile=/etc/httpd/conf/httpd.conf op monitor interval=1min[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness="100"
[root@pcmk-1 ~]# crm_mon
============
Last updated: Fri Aug 28 16:12:49 2009
Stack: openais
Current DC: pcmk-2 - partition with quorum
Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f
2 Nodes configured, 2 expected votes
2 Resources configured.
============
Online: [ pcmk-1 pcmk-2 ]
ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2
WebSite (ocf::heartbeat:apache): Started pcmk-1
[root@pcmk-1 ~]#crm configure colocation website-with-ip INFINITY: WebSite ClusterIP[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" colocation website-with-ip inf: WebSite ClusterIP property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness="100" [root@pcmk-1 ~]#crm_mon============ Last updated: Fri Aug 28 16:14:34 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 2 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2
[root@pcmk-1 ~]#crm configure order apache-after-ip mandatory: ClusterIP WebSite[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" colocation website-with-ip inf: WebSite ClusterIP order apache-after-ip inf: ClusterIP WebSite property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness="100"
[root@pcmk-1 ~]#crm configure location prefer-pcmk-1 WebSite 50: pcmk-1[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" location prefer-pcmk-1 WebSite 50: pcmk-1 colocation website-with-ip inf: WebSite ClusterIP property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness="100" [root@pcmk-1 ~]#crm_mon============ Last updated: Fri Aug 28 16:17:35 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 2 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2
[root@pcmk-1 ~]#crm resource move WebSite pcmk-1[root@pcmk-1 ~]#crm_mon============ Last updated: Fri Aug 28 16:19:24 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 2 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1 Notice how the colocation rule we created has ensured that ClusterIP was also moved to pcmk-1. For the curious, we can see the effect of this command by examining the configuration crm configure show [root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" location cli-prefer-WebSite WebSite \ rule $id="cli-prefer-rule-WebSite" inf: #uname eq pcmk-1 location prefer-pcmk-1 WebSite 50: pcmk-1 colocation website-with-ip inf: WebSite ClusterIP property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness="100"
[root@pcmk-1 ~]#crm resource unmove WebSite[root@pcmk-1 ~]#crm configure shownode pcmk-1 node pcmk-2 primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" location prefer-pcmk-1 WebSite 50: pcmk-1 colocation website-with-ip inf: WebSite ClusterIP property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness="100"
[root@pcmk-1 ~]# crm_mon
============
Last updated: Fri Aug 28 16:20:53 2009
Stack: openais
Current DC: pcmk-2 - partition with quorum
Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f
2 Nodes configured, 2 expected votes
2 Resources configured.
============
Online: [ pcmk-1 pcmk-2 ]
ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1
WebSite (ocf::heartbeat:apache): Started pcmk-1
[15] Compare the key used here ocf:heartbeat:apache with the one we used earlier for the IP address: ocf:heartbeat:IPaddr2
[root@pcmk-1 ~]# yum install -y drbd-pacemaker
Loaded plugins: presto, refresh-packagekit
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package drbd-pacemaker.x86_64 0:8.3.7-2.fc13 set to be updated
--> Processing Dependency: drbd-utils = 8.3.7-2.fc13 for package: drbd-pacemaker-8.3.7-2.fc13.x86_64
--> Running transaction check
---> Package drbd-utils.x86_64 0:8.3.7-2.fc13 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
=================================================================================
Package Arch Version Repository Size
=================================================================================
Installing:
drbd-pacemaker x86_64 8.3.7-2.fc13 fedora 19 k
Installing for dependencies:
drbd-utils x86_64 8.3.7-2.fc13 fedora 165 k
Transaction Summary
=================================================================================
Install 2 Package(s)
Upgrade 0 Package(s)
Total download size: 184 k
Installed size: 427 k
Downloading Packages:
Setting up and reading Presto delta metadata
fedora/prestodelta | 1.7 kB 00:00
Processing delta metadata
Package(s) data still to download: 184 k
(1/2): drbd-pacemaker-8.3.7-2.fc13.x86_64.rpm | 19 kB 00:01
(2/2): drbd-utils-8.3.7-2.fc13.x86_64.rpm | 165 kB 00:02
---------------------------------------------------------------------------------
Total 45 kB/s | 184 kB 00:04
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : drbd-utils-8.3.7-2.fc13.x86_64 1/2
Installing : drbd-pacemaker-8.3.7-2.fc13.x86_64 2/2
Installed:
drbd-pacemaker.x86_64 0:8.3.7-2.fc13
Dependency Installed:
drbd-utils.x86_64 0:8.3.7-2.fc13
Complete!
[root@pcmk-1 ~]#
[root@pcmk-1 ~]#lvcreate -n drbd-demo -L 1G VolGroupLogical volume "drbd-demo" created [root@pcmk-1 ~]#lvsLV VG Attr LSize Origin Snap% Move Log Copy% Convert drbd-demo VolGroup -wi-a- 1.00G lv_root VolGroup -wi-ao 7.30G lv_swap VolGroup -wi-ao 500.00M
[root@pcmk-2 ~]#lvsLV VG Attr LSize Origin Snap% Move Log Copy% Convert lv_root VolGroup -wi-ao 7.30G lv_swap VolGroup -wi-ao 500.00M [root@pcmk-2 ~]#lvcreate -n drbd-demo -L 1G VolGroupLogical volume "drbd-demo" created [root@pcmk-2 ~]#lvsLV VG Attr LSize Origin Snap% Move Log Copy% Convert drbd-demo VolGroup -wi-a- 1.00G lv_root VolGroup -wi-ao 7.30G lv_swap VolGroup -wi-ao 500.00M
global {
usage-count yes;
}
common {
protocol C;
}
resource wwwdata {
meta-disk internal;
device /dev/drbd1;
syncer {
verify-alg sha1;
}
net {
allow-two-primaries;
}
on pcmk-1 {
disk /dev/mapper/VolGroup-drbd--demo;
address 192.168.122.101:7789;
}
on
pcmk-2 {
disk /dev/mapper/VolGroup-drbd--demo;
address 192.168.122.102:7789;
}
}
[root@pcmk-1 ~]#drbdadm create-md wwwdatamd_offset 12578816 al_offset 12546048 bm_offset 12541952 Found some data ==> This might destroy existing data! <== Do you want to proceed? [need to type 'yes' to confirm]yesWriting meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. success
[root@pcmk-1 ~]#modprobe drbd[root@pcmk-1 ~]#drbdadm up wwwdata[root@pcmk-1 ~]#cat /proc/drbdversion: 8.3.6 (api:88/proto:86-90) GIT-hash: f3606c47cc6fcf6b3f086e425cb34af8b7a81bbf build by root@pcmk-1, 2009-12-08 11:22:57 1: cs:WFConnection ro:Secondary/Unknown ds:Inconsistent/DUnknown C r---- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:12248 [root@pcmk-1 ~]# Repeat on the second node drbdadm --force create-md wwwdata modprobe drbd drbdadm up wwwdata cat /proc/drbd [root@pcmk-2 ~]#drbdadm --force create-md wwwdataWriting meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. success [root@pcmk-2 ~]#modprobe drbdWARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. [root@pcmk-2 ~]#drbdadm up wwwdata[root@pcmk-2 ~]#cat /proc/drbdversion: 8.3.6 (api:88/proto:86-90) GIT-hash: f3606c47cc6fcf6b3f086e425cb34af8b7a81bbf build by root@pcmk-1, 2009-12-08 11:22:57 1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r---- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:12248
[root@pcmk-1 ~]#drbdadm -- --overwrite-data-of-peer primary wwwdata[root@pcmk-1 ~]#cat /proc/drbdversion: 8.3.6 (api:88/proto:86-90) GIT-hash: f3606c47cc6fcf6b3f086e425cb34af8b7a81bbf build by root@pcmk-1, 2009-12-08 11:22:57 1: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---- ns:2184 nr:0 dw:0 dr:2472 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:10064 [=====>..............] sync'ed: 33.4% (10064/12248)K finish: 0:00:37 speed: 240 (240) K/sec [root@pcmk-1 ~]#cat /proc/drbdversion: 8.3.6 (api:88/proto:86-90) GIT-hash: f3606c47cc6fcf6b3f086e425cb34af8b7a81bbf build by root@pcmk-1, 2009-12-08 11:22:57 1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---- ns:12248 nr:0 dw:0 dr:12536 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root@pcmk-1 ~]#mkfs.ext4 /dev/drbd1mke2fs 1.41.4 (27-Jan-2009) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 3072 inodes, 12248 blocks 612 blocks (5.00%) reserved for the super user First data block=1 Maximum filesystem blocks=12582912 2 block groups 8192 blocks per group, 8192 fragments per group 1536 inodes per group Superblock backups stored on blocks: 8193 Writing inode tables: done Creating journal (1024 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 26 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. Now mount the newly created filesystem so we can create our index file mount /dev/drbd1 /mnt/ cat <<-END >/mnt/index.html <html> <body>My Test Site - drbd</body> </html> END umount /dev/drbd1 [root@pcmk-1 ~]#mount /dev/drbd1 /mnt/[root@pcmk-1 ~]#cat <<-END >/mnt/index.html> <html> > <body>My Test Site - drbd</body> > </html> > END [root@pcmk-1 ~]#umount /dev/drbd1
[root@pcmk-1 ~]# crm
cib crm(live)#
cib crm(live)# cib new drbd
INFO: drbd shadow CIB created
crm(drbd)#
crm(drbd)#configure primitive WebData ocf:linbit:drbd params drbd_resource=wwwdata \op monitor interval=60scrm(drbd)#configure ms WebDataClone WebData meta master-max=1 master-node-max=1 \clone-max=2 clone-node-max=1 notify=truecrm(drbd)#configure shownode pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" ms WebDataClone WebData \ meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" location prefer-pcmk-1 WebSite 50: pcmk-1 colocation website-with-ip inf: WebSite ClusterIP order apache-after-ip inf: ClusterIP WebSite property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
crm(drbd)#cib commit drbdINFO: commited 'drbd' shadow CIB to the cluster crm(drbd)#quitbye [root@pcmk-1 ~]#crm_mon============ Last updated: Tue Sep 1 09:37:13 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 3 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1 Master/Slave Set: WebDataClone Masters: [ pcmk-2 ] Slaves: [ pcmk-1 ]
[root@pcmk-1 ~]#crmcrm(live)#cib new fsINFO: fs shadow CIB created crm(fs)#configure primitive WebFS ocf:heartbeat:Filesystem \params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4"crm(fs)#configure colocation fs_on_drbd inf: WebFS WebDataClone:Mastercrm(fs)#configure order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
crm(fs)#configure colocation WebSite-with-WebFS inf: WebSite WebFScrm(fs)#configure order WebSite-after-WebFS inf: WebFS WebSite
[root@pcmk-1 ~]# crm configure show
node pcmk-1
node pcmk-2
primitive WebData ocf:linbit:drbd \
params drbd_resource="wwwdata" \
op monitor interval="60s"
primitive WebFS ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4"
primitive WebSite ocf:heartbeat:apache \
params configfile="/etc/httpd/conf/httpd.conf" \
op monitor interval="1min"
primitive ClusterIP ocf:heartbeat:IPaddr2 \
params ip="192.168.122.101" cidr_netmask="32" \
op monitor interval="30s"
ms WebDataClone WebData \
meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
location prefer-pcmk-1 WebSite 50: pcmk-1
colocation WebSite-with-WebFS inf: WebSite WebFS
colocation fs_on_drbd inf: WebFS WebDataClone:Master
colocation website-with-ip inf: WebSite ClusterIP
order WebFS-after-WebData inf: WebDataClone:promote WebFS:start
order WebSite-after-WebFS inf: WebFS WebSite
order apache-after-ip inf: ClusterIP WebSite
property $id="cib-bootstrap-options" \
dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
cluster-infrastructure="openais" \
expected-quorum-votes=”2” \
stonith-enabled="false" \
no-quorum-policy="ignore"
rsc_defaults $id="rsc-options" \
resource-stickiness=”100”
crm(fs)#cib commit fsINFO: commited 'fs' shadow CIB to the cluster crm(fs)#quitbye [root@pcmk-1 ~]#crm_mon============ Last updated: Tue Sep 1 10:08:44 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 4 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1 Master/Slave Set: WebDataClone Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
[root@pcmk-1 ~]#crm node standby[root@pcmk-1 ~]#crm_mon============ Last updated: Tue Sep 1 10:09:57 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 4 Resources configured. ============ Node pcmk-1: standby Online: [ pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone Masters: [ pcmk-2 ] Stopped: [ WebData:1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
[root@pcmk-1 ~]#crm node online[root@pcmk-1 ~]#crm_mon============ Last updated: Tue Sep 1 10:13:25 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 4 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone Masters: [ pcmk-2 ] Slaves: [ pcmk-1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
[root@pcmk-1 ~]# yum install -y gfs2-utils gfs-pcmk
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package gfs-pcmk.x86_64 0:3.0.5-2.fc12 set to be updated
--> Processing Dependency: libSaCkpt.so.3(OPENAIS_CKPT_B.01.01)(64bit) for package: gfs-pcmk-3.0.5-2.fc12.x86_64
--> Processing Dependency: dlm-pcmk for package: gfs-pcmk-3.0.5-2.fc12.x86_64
--> Processing Dependency: libccs.so.3()(64bit) for package: gfs-pcmk-3.0.5-2.fc12.x86_64
--> Processing Dependency: libdlmcontrol.so.3()(64bit) for package: gfs-pcmk-3.0.5-2.fc12.x86_64
--> Processing Dependency: liblogthread.so.3()(64bit) for package: gfs-pcmk-3.0.5-2.fc12.x86_64
--> Processing Dependency: libSaCkpt.so.3()(64bit) for package: gfs-pcmk-3.0.5-2.fc12.x86_64
---> Package gfs2-utils.x86_64 0:3.0.5-2.fc12 set to be updated
--> Running transaction check
---> Package clusterlib.x86_64 0:3.0.5-2.fc12 set to be updated
---> Package dlm-pcmk.x86_64 0:3.0.5-2.fc12 set to be updated
---> Package openaislib.x86_64 0:1.1.0-1.fc12 set to be updated
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================
Package Arch Version Repository Size
===========================================================================================
Installing:
gfs-pcmk x86_64 3.0.5-2.fc12 custom 101 k
gfs2-utils x86_64 3.0.5-2.fc12 custom 208 k
Installing for dependencies:
clusterlib x86_64 3.0.5-2.fc12 custom 65 k
dlm-pcmk x86_64 3.0.5-2.fc12 custom 93 k
openaislib x86_64 1.1.0-1.fc12 fedora 76 k
Transaction Summary
===========================================================================================
Install 5 Package(s)
Upgrade 0 Package(s)
Total download size: 541 k
Downloading Packages:
(1/5): clusterlib-3.0.5-2.fc12.x86_64.rpm | 65 kB 00:00
(2/5): dlm-pcmk-3.0.5-2.fc12.x86_64.rpm | 93 kB 00:00
(3/5): gfs-pcmk-3.0.5-2.fc12.x86_64.rpm | 101 kB 00:00
(4/5): gfs2-utils-3.0.5-2.fc12.x86_64.rpm | 208 kB 00:00
(5/5): openaislib-1.1.0-1.fc12.x86_64.rpm | 76 kB 00:00
-------------------------------------------------------------------------------------------
Total 992 kB/s | 541 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : clusterlib-3.0.5-2.fc12.x86_64 1/5
Installing : openaislib-1.1.0-1.fc12.x86_64 2/5
Installing : dlm-pcmk-3.0.5-2.fc12.x86_64 3/5
Installing : gfs-pcmk-3.0.5-2.fc12.x86_64 4/5
Installing : gfs2-utils-3.0.5-2.fc12.x86_64 5/5
Installed:
gfs-pcmk.x86_64 0:3.0.5-2.fc12 gfs2-utils.x86_64 0:3.0.5-2.fc12
Dependency Installed:
clusterlib.x86_64 0:3.0.5-2.fc12 dlm-pcmk.x86_64 0:3.0.5-2.fc12
openaislib.x86_64 0:1.1.0-1.fc12
Complete!
[root@pcmk-1 x86_64]#
dlm_controld.pcmk and gfs_controld.pcmk and live in the /usr/sbin directory.
cman and reconfigure Corosync to use it as outlined in Appendix C, Using CMAN for Cluster Membership and Quorum.
dlm-clone and gfs-clone are created, and proceed directly to Section 8.4, “Create a GFS2 Filesystem”.
[root@pcmk-1 ~]#crmcrm(live)#cib new stack-glueINFO: stack-glue shadow CIB created crm(stack-glue)#configure primitive dlm ocf:pacemaker:controld op monitor interval=120scrm(stack-glue)#configure clone dlm-clone dlm meta interleave=truecrm(stack-glue)#configure show xmlcrm(stack-glue)#configure shownode pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4" primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" ms WebDataClone WebData \ meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" clone dlm-clone dlm \ meta interleave="true" location prefer-pcmk-1 WebSite 50: pcmk-1 colocation WebSite-with-WebFS inf: WebSite WebFS colocation fs_on_drbd inf: WebFS WebDataClone:Master colocation website-with-ip inf: WebSite ClusterIP order WebFS-after-WebData inf: WebDataClone:promote WebFS:start order WebSite-after-WebFS inf: WebFS WebSite order apache-after-ip inf: ClusterIP WebSite property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
crm(stack-glue)#cib commit stack-glueINFO: commited 'stack-glue' shadow CIB to the cluster crm(stack-glue)#quitbye [root@pcmk-1 ~]#crm_mon============ Last updated: Thu Sep 3 20:49:54 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 5 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 Clone Set: dlm-clone Started: [ pcmk-2 pcmk-1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
[root@pcmk-1 ~]#crmcrm(live)#cib new gfs-glue --forceINFO: gfs-glue shadow CIB created crm(gfs-glue)#configure primitive gfs-control ocf:pacemaker:controld params daemon=gfs_controld.pcmk args="-g 0" op monitor interval=120scrm(gfs-glue)#configure clone gfs-clone gfs-control meta interleave=true
crm(gfs-glue)#configure colocation gfs-with-dlm INFINITY: gfs-clone dlm-clonecrm(gfs-glue)#configure order start-gfs-after-dlm mandatory: dlm-clone gfs-clone
crm(gfs-glue)#configure shownode pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype="ext4" primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" primitive gfs-control ocf:pacemaker:controld \ params daemon=”gfs_controld.pcmk” args=”-g 0” \ op monitor interval="120s" ms WebDataClone WebData \ meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" clone dlm-clone dlm \ meta interleave="true" clone gfs-clone gfs-control \ meta interleave="true" location prefer-pcmk-1 WebSite 50: pcmk-1 colocation WebSite-with-WebFS inf: WebSite WebFS colocation fs_on_drbd inf: WebFS WebDataClone:Master colocation gfs-with-dlm inf: gfs-clone dlm-clone colocation website-with-ip inf: WebSite ClusterIP order WebFS-after-WebData inf: WebDataClone:promote WebFS:start order WebSite-after-WebFS inf: WebFS WebSite order apache-after-ip inf: ClusterIP WebSite order start-gfs-after-dlm inf: dlm-clone gfs-clone property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100” crm(gfs-glue)#cib commit gfs-glueINFO: commited 'gfs-glue' shadow CIB to the cluster crm(gfs-glue)#quitbye [root@pcmk-1 ~]#crm_mon============ Last updated: Thu Sep 3 20:49:54 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 6 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 Clone Set: dlm-clone Started: [ pcmk-2 pcmk-1 ] Clone Set: gfs-clone Started: [ pcmk-2 pcmk-1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
[root@pcmk-1 ~]#crm_resource --resource WebFS --set-parameter target-role --meta --parameter-value Stopped[root@pcmk-1 ~]#crm_mon============ Last updated: Thu Sep 3 15:18:06 2009 Stack: openais Current DC: pcmk-1 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 6 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] Master/Slave Set: WebDataClone Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-1 Clone Set: dlm-clone Started: [ pcmk-2 pcmk-1 ] Clone Set: gfs-clone Started: [ pcmk-2 pcmk-1 ]
mkfs.gfs2 -p lock_dlm -j 2 -t pcmk:web /dev/drbd1
[root@pcmk-1 ~]# mkfs.gfs2 -t pcmk:web -p lock_dlm -j 2 /dev/vdb
This will destroy any data on /dev/vdb.
It appears to contain: data
Are you sure you want to proceed? [y/n] y
Device: /dev/vdb
Blocksize: 4096
Device Size 1.00 GB (131072 blocks)
Filesystem Size: 1.00 GB (131070 blocks)
Journals: 2
Resource Groups: 2
Locking Protocol: "lock_dlm"
Lock Table: "pcmk:web"
UUID: 6B776F46-177B-BAF8-2C2B-292C0E078613
[root@pcmk-1 ~]#
[root@pcmk-1 ~]#mount /dev/drbd1 /mnt/[root@pcmk-1 ~]#cat <<-END >/mnt/index.html<html> <body>My Test Site - GFS2</body> </html> END [root@pcmk-1 ~]#umount /dev/drbd1[root@pcmk-1 ~]#drbdadm verify wwwdata[root@pcmk-1 ~]#
[root@pcmk-1 ~]#crmcrm(live)#cib new GFS2INFO: GFS2 shadow CIB created crm(GFS2)#configure delete WebFScrm(GFS2)#configure primitive WebFS ocf:heartbeat:Filesystem params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2”
crm(GFS2)#configure colocation WebSite-with-WebFS inf: WebSite WebFScrm(GFS2)#configure colocation fs_on_drbd inf: WebFS WebDataClone:Mastercrm(GFS2)#configure order WebFS-after-WebData inf: WebDataClone:promote WebFS:startcrm(GFS2)#configure order WebSite-after-WebFS inf: WebFS WebSitecrm(GFS2)#configure colocation WebFS-with-gfs-control INFINITY: WebFS gfs-clonecrm(GFS2)#configure order start-WebFS-after-gfs-control mandatory: gfs-clone WebFScrm(GFS2)#configure shownode pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2” primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" \ op monitor interval="30s" primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" primitive gfs-control ocf:pacemaker:controld \ params daemon=”gfs_controld.pcmk” args=”-g 0” \ op monitor interval="120s" ms WebDataClone WebData \ meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" clone dlm-clone dlm \ meta interleave="true" clone gfs-clone gfs-control \ meta interleave="true" colocation WebFS-with-gfs-control inf: WebFS gfs-clone colocation WebSite-with-WebFS inf: WebSite WebFS colocation fs_on_drbd inf: WebFS WebDataClone:Master colocation gfs-with-dlm inf: gfs-clone dlm-clone colocation website-with-ip inf: WebSite ClusterIP order WebFS-after-WebData inf: WebDataClone:promote WebFS:start order WebSite-after-WebFS inf: WebFS WebSite order apache-after-ip inf: ClusterIP WebSite order start-WebFS-after-gfs-control inf: gfs-clone WebFS order start-gfs-after-dlm inf: dlm-clone gfs-clone property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
crm(GFS2)#cib commit GFS2INFO: commited 'GFS2' shadow CIB to the cluster crm(GFS2)#quitbye [root@pcmk-1 ~]#crm_mon============ Last updated: Thu Sep 3 20:49:54 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 6 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] ClusterIP (ocf::heartbeat:IPaddr): Started pcmk-2 Clone Set: dlm-clone Started: [ pcmk-2 pcmk-1 ] Clone Set: gfs-clone Started: [ pcmk-2 pcmk-1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
[root@pcmk-1 ~]#crm[root@pcmk-1 ~]#cib new active
[root@pcmk-1 ~]#configure clone WebIP ClusterIP \meta globally-unique=”true” clone-max=”2” clone-node-max=”2”
[root@pcmk-1 ~]# configure edit ClusterIP
clusterip_hash="sourceip"
primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip="192.168.122.101" cidr_netmask="32" clusterip_hash="sourceip" \ op monitor interval="30s"
[root@pcmk-1 ~]#crmcrm(live)#cib new activeINFO: active shadow CIB created crm(active)#configure clone WebIP ClusterIP \meta globally-unique=”true” clone-max=”2” clone-node-max=”2” crm(active)#configure shownode pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2” primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip=”192.168.122.101” cidr_netmask=”32” clusterip_hash=”sourceip” \ op monitor interval="30s" primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" primitive gfs-control ocf:pacemaker:controld \ params daemon=”gfs_controld.pcmk” args=”-g 0” \ op monitor interval="120s" ms WebDataClone WebData \ meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" clone WebIP ClusterIP \ meta globally-unique=”true” clone-max=”2” clone-node-max=”2” clone dlm-clone dlm \ meta interleave="true" clone gfs-clone gfs-control \ meta interleave="true" colocation WebFS-with-gfs-control inf: WebFS gfs-clone colocation WebSite-with-WebFS inf: WebSite WebFS colocation fs_on_drbd inf: WebFS WebDataClone:Master colocation gfs-with-dlm inf: gfs-clone dlm-clone colocation website-with-ip inf: WebSite WebIP order WebFS-after-WebData inf: WebDataClone:promote WebFS:start order WebSite-after-WebFS inf: WebFS WebSite order apache-after-ip inf: WebIP WebSite order start-WebFS-after-gfs-control inf: gfs-clone WebFS order start-gfs-after-dlm inf: dlm-clone gfs-clone property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled="false" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
crm(active)#configure clone WebFSClone WebFScrm(active)#configure clone WebSiteClone WebSite
crm(active)# configure edit WebDataClone
crm(active)# configure show
node pcmk-1
node pcmk-2
primitive WebData ocf:linbit:drbd \
params drbd_resource="wwwdata" \
op monitor interval="60s"
primitive WebFS ocf:heartbeat:Filesystem \
params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2”
primitive WebSite ocf:heartbeat:apache \
params configfile="/etc/httpd/conf/httpd.conf" \
op monitor interval="1min"
primitive ClusterIP ocf:heartbeat:IPaddr2 \
params ip=”192.168.122.101” cidr_netmask=”32” clusterip_hash=”sourceip” \
op monitor interval="30s"
primitive dlm ocf:pacemaker:controld \
op monitor interval="120s"
primitive gfs-control ocf:pacemaker:controld \
params daemon=”gfs_controld.pcmk” args=”-g 0” \
op monitor interval="120s"
ms WebDataClone WebData \
meta master-max="2" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
clone WebFSClone WebFS
clone WebIP ClusterIP \
meta globally-unique=”true” clone-max=”2” clone-node-max=”2”
clone WebSiteClone WebSite
clone dlm-clone dlm \
meta interleave="true"
clone gfs-clone gfs-control \
meta interleave="true"
colocation WebFS-with-gfs-control inf: WebFSClone gfs-clone
colocation WebSite-with-WebFS inf: WebSiteClone WebFSClone
colocation fs_on_drbd inf: WebFSClone WebDataClone:Master
colocation gfs-with-dlm inf: gfs-clone dlm-clone
colocation website-with-ip inf: WebSiteClone WebIP
order WebFS-after-WebData inf: WebDataClone:promote WebFSClone:start
order WebSite-after-WebFS inf: WebFSClone WebSiteClone
order apache-after-ip inf: WebIP WebSiteClone
order start-WebFS-after-gfs-control inf: gfs-clone WebFSClone
order start-gfs-after-dlm inf: dlm-clone gfs-clone
property $id="cib-bootstrap-options" \
dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \
cluster-infrastructure="openais" \
expected-quorum-votes=”2” \
stonith-enabled="false" \
no-quorum-policy="ignore"
rsc_defaults $id="rsc-options" \
resource-stickiness=”100”
crm(active)#cib commit activeINFO: commited 'active' shadow CIB to the cluster crm(active)#quitbye [root@pcmk-1 ~]#crm_mon============ Last updated: Thu Sep 3 21:37:27 2009 Stack: openais Current DC: pcmk-2 - partition with quorum Version: 1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f 2 Nodes configured, 2 expected votes 6 Resources configured. ============ Online: [ pcmk-1 pcmk-2 ] Master/Slave Set: WebDataClone Masters: [ pcmk-1 pcmk-2 ] Clone Set: dlm-clone Started: [ pcmk-2 pcmk-1 ] Clone Set: gfs-clone Started: [ pcmk-2 pcmk-1 ] Clone Set: WebIP Started: [ pcmk-1 pcmk-2 ] Clone Set: WebFSClone Started: [ pcmk-1 pcmk-2 ] Clone Set: WebSiteClone Started: [ pcmk-1 pcmk-2 ]
lrmadmin -M stonith {type} pacemaker
stonith -t external/ibmrsa -n
[root@pcmk-1 ~]# stonith -t external/ibmrsa -n
hostname ipaddr userid passwd type
[root@pcmk-1 ~]#crmcrm(live)#cib new stonithINFO: stonith shadow CIB created crm(stonith)#configure primitive rsa-fencing stonith::external/ibmrsa \params hostname=”pcmk-1 pcmk-2" ipaddr=192.168.122.31 userid=mgmt passwd=abc123 type=ibm \op monitor interval="60s"crm(stonith)#configure clone Fencing rsa-fencing
crm(stonith)#configure property stonith-enabled="true"crm(stonith)#configure shownode pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2” primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip=”192.168.122.101” cidr_netmask=”32” clusterip_hash=”sourceip” \ op monitor interval="30s" primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" primitive gfs-control ocf:pacemaker:controld \ params daemon=”gfs_controld.pcmk” args=”-g 0” \ op monitor interval="120s" primitive rsa-fencing stonith::external/ibmrsa \ params hostname=”pcmk-1 pcmk-2" ipaddr=192.168.122.31 userid=mgmt passwd=abc123 type=ibm \ op monitor interval="60s" ms WebDataClone WebData \ meta master-max="2" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" clone Fencing rsa-fencing clone WebFSClone WebFS clone WebIP ClusterIP \ meta globally-unique=”true” clone-max=”2” clone-node-max=”2” clone WebSiteClone WebSite clone dlm-clone dlm \ meta interleave="true" clone gfs-clone gfs-control \ meta interleave="true" colocation WebFS-with-gfs-control inf: WebFSClone gfs-clone colocation WebSite-with-WebFS inf: WebSiteClone WebFSClone colocation fs_on_drbd inf: WebFSClone WebDataClone:Master colocation gfs-with-dlm inf: gfs-clone dlm-clone colocation website-with-ip inf: WebSiteClone WebIP order WebFS-after-WebData inf: WebDataClone:promote WebFSClone:start order WebSite-after-WebFS inf: WebFSClone WebSiteClone order apache-after-ip inf: WebIP WebSiteClone order start-WebFS-after-gfs-control inf: gfs-clone WebFSClone order start-gfs-after-dlm inf: dlm-clone gfs-clone property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled="true" \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
[root@pcmk-1 ~]# crm configure show node pcmk-1 node pcmk-2 primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2” primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip=”192.168.122.101” cidr_netmask=”32” clusterip_hash=”sourceip” \ op monitor interval="30s" primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" primitive gfs-control ocf:pacemaker:controld \ params daemon=”gfs_controld.pcmk” args=”-g 0” \ op monitor interval="120s" primitive rsa-fencing stonith::external/ibmrsa \ params hostname=”pcmk-1 pcmk-2" ipaddr=192.168.122.31 userid=mgmt passwd=abc123 type=ibm \ op monitor interval="60s" ms WebDataClone WebData \ meta master-max="2" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" clone Fencing rsa-fencing clone WebFSClone WebFS clone WebIP ClusterIP \ meta globally-unique=”true” clone-max=”2” clone-node-max=”2” clone WebSiteClone WebSite clone dlm-clone dlm \ meta interleave="true" clone gfs-clone gfs-control \ meta interleave="true" colocation WebFS-with-gfs-control inf: WebFSClone gfs-clone colocation WebSite-with-WebFS inf: WebSiteClone WebFSClone colocation fs_on_drbd inf: WebFSClone WebDataClone:Master colocation gfs-with-dlm inf: gfs-clone dlm-clone colocation website-with-ip inf: WebSiteClone WebIP order WebFS-after-WebData inf: WebDataClone:promote WebFSClone:start order WebSite-after-WebFS inf: WebFSClone WebSiteClone order apache-after-ip inf: WebIP WebSiteClone order start-WebFS-after-gfs-control inf: gfs-clone WebFSClone order start-gfs-after-dlm inf: dlm-clone gfs-clone property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled=”true” \ no-quorum-policy="ignore" rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
node pcmk-1 node pcmk-2
property $id="cib-bootstrap-options" \ dc-version="1.1.5-bdd89e69ba545404d02445be1f3d72e6a203ba2f" \ cluster-infrastructure="openais" \ expected-quorum-votes=”2” \ stonith-enabled=”true” \ no-quorum-policy="ignore"
rsc_defaults $id="rsc-options" \ resource-stickiness=”100”
primitive rsa-fencing stonith::external/ibmrsa \ params hostname=”pcmk-1 pcmk-2" ipaddr=192.168.122.31 userid=mgmt passwd=abc123 type=ibm \ op monitor interval="60s" clone Fencing rsa-fencing
primitive ClusterIP ocf:heartbeat:IPaddr2 \ params ip=”192.168.122.101” cidr_netmask=”32” clusterip_hash=”sourceip” \ op monitor interval="30s" clone WebIP ClusterIP meta globally-unique=”true” clone-max=”2” clone-node-max=”2”
primitive dlm ocf:pacemaker:controld \ op monitor interval="120s" clone dlm-clone dlm \ meta interleave="true
interleave is no longer needed
primitive gfs-control ocf:pacemaker:controld \ params daemon=”gfs_controld.pcmk” args=”-g 0” \ op monitor interval="120s" clone gfs-clone gfs-control \ meta interleave="true" colocation gfs-with-dlm inf: gfs-clone dlm-clone order start-gfs-after-dlm inf: dlm-clone gfs-clone
primitive WebData ocf:linbit:drbd \ params drbd_resource="wwwdata" \ op monitor interval="60s" ms WebDataClone WebData \ meta master-max="2" master-node-max="1" clone-max="2" clone-node-max="1" notify="true"
primitive WebFS ocf:heartbeat:Filesystem \ params device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" fstype=”gfs2” clone WebFSClone WebFS colocation WebFS-with-gfs-control inf: WebFSClone gfs-clone colocation fs_on_drbd inf: WebFSClone WebDataClone:Master order WebFS-after-WebData inf: WebDataClone:promote WebFSClone:start order start-WebFS-after-gfs-control inf: gfs-clone WebFSClone
primitive WebSite ocf:heartbeat:apache \ params configfile="/etc/httpd/conf/httpd.conf" \ op monitor interval="1min" clone WebSiteClone WebSite colocation WebSite-with-WebFS inf: WebSiteClone WebFSClone colocation website-with-ip inf: WebSiteClone WebIP order apache-after-ip inf: WebIP WebSiteClone order WebSite-after-WebFS inf: WebFSClone WebSiteClone
# Please read the Corosync.conf.5 manual page
compatibility: whitetank
totem {
version: 2
# How long before declaring a token lost (ms)
token: 5000
# How many token retransmits before forming a new configuration
token_retransmits_before_loss_const: 10
# How long to wait for join messages in the membership protocol (ms)
join: 1000
# How long to wait for consensus to be achieved before starting a new
# round of membership configuration (ms)
consensus: 6000
# Turn off the virtual synchrony filter
vsftype: none
# Number of messages that may be sent by one processor on receipt of the token
max_messages: 20
# Stagger sending the node join messages by 1..send_join ms
send_join: 45
# Limit generated nodeids to 31-bits (positive signed integers)
clear_node_high_bit: yes
# Disable encryption
secauth: off
# How many threads to use for encryption/decryption
threads: 0
# Optionally assign a fixed node id (integer)
# nodeid: 1234
interface {
ringnumber: 0
# The following values need to be set based on your environment
bindnetaddr: 192.168.122.0
mcastaddr: 226.94.1.1
mcastport: 4000
}
}
logging {
debug: off
fileline: off
to_syslog: yes
to_stderr: off
syslog_facility: daemon
timestamp: on
}
amf {
mode: disabled
}
/etc/corosync/service.d/pcmk and stopping Corosync.
cluster.conf and use the /etc/init.d/cman script to start Corosync. Its far easier to maintain and start automatically starts the necessary pieces for using GFS2.
Fencing, Failover Domains, or HA Services and anything to do with rgmanager and fenced. All these continue to be handled by Pacemaker in the normal manner.
<?xml version="1.0"?>
<cluster config_version="1" name="beekhof">
<fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
<clusternodes>
<clusternode name="pcmk-1" nodeid="1">
<fence/>
</clusternode>
<clusternode name="pcmk-2" nodeid="2">
<fence/>
</clusternode>
</clusternodes>
<cman/>
<fencedevices/>
<rm/>
</cluster>
corosync.conf. We recommend you place these directives in /etc/corosync/service.d/cman as they will differ between machines.
[root@pcmk-1 ~]# cat <<-END >>/etc/corosync/service.d/cman
cluster {
name: beekhof
clusternodes {
clusternode {
votes: 1
nodeid: 1
name: pcmk-1
}
clusternode {
votes: 1
nodeid: 2
name: pcmk-2
}
}
cman {
expected_votes: 2
cluster_id: 123
nodename: `uname -n`
two_node: 1
max_queued: 10
}
}
service {
name: corosync_cman
ver: 0
}
quorum {
provider: quorum_cman
}
END
nodename was set appropriately on each host.
[17]
A failure to do this can lead to what is called internal split-brain - a situation where different parts of the stack disagree about whether some nodes are alive or dead - which quickly leads to unnecssary down-time and/or data corruption.
| Revision History | |||
|---|---|---|---|
| Revision 1 | Mon May 17 2010 | ||
| |||